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ABSTRACT 

TOPS, a set of materials designed\to improve problem 
solving instruction, was evaluated in two large urban school 
districts from Fall 1980' to Spring 1982. At Site A, the program was 
implemented at. the middle school level (grades 5-8), where most 
mathematics teachers were specialists; they accepted the, 
problem-solving activities after some initial hesitation. At Site B, 
the program was implemented in grades 3-8, though most often in 
grades 4 and 5, usually by teachers who were not mathematics 
Specialists. They varied in their belief that^the activities "would 
improve problem-solving ability, and, even though in year 2 they were 
fallowed to develop lessons involving problem-solving strategies tied 
mote closely to textbpoks and tests, problem solving was never 

> completely accepted by all teachers. At Site A, 80 teachers were 
trained, eventually affecting over 300 classes, and the school board * 

. pledged continued support. At Site B, progress was slower. Two major 
effects were noted at both sites: (1) teachers viewed TOPS favorably 
and commented on its positive impact on students 1 mathematics and 
thinking skills; and (2> TOPS students had better test scores than 
non-TOPS students; differences were significant at Site A but not 
Site B. (MNS) * ' 
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' ' SUMMARY^ 

" TOPS goals were twofold: 1 to train a corps of teachers capable of teaching 
problem solving as well as training other teachers to do the same, and to 
improve students' problem solving in mathematics. 

Summary of Implementation.. TOPS was implemented differently at the two sites. 
At Site kj the program was implemented at the middle school level (grades 5 - 
8), where most math teachers were specialists who teach mathematics throughout 
the day. For the most part, the original group of TOPS teachers was a 
relatively strong group which might^be expected to take a leadership role in 
future \0PS expansion. After some hesitation, mathematical problem solving as 
exemplified by the. f OPS activities was accepted at Site A.' This was in spite 

t 

of .the fact that this problem solving was never clearly articulated as 
measurable and seemed not particularly consonant with the district's 
standardized tests. 

A^Site B, the program was implemented at grades 3-8, though most often 
in grades four and five, and usually .by teachers who were not mathematics 
specialists. The most important criterion for selecting a schdol was low 
scores for the school on district administered achievement tests. The 

■i 

original group of teachers did not have a particularly strong background in 
mathematics and participants varied in their belief that JOPS activities would 
lead to improved problem solving ability, especially as measured in 
standardized tests. A redirection of the program in"Year II at Site B 
broadened and diffused the instructional emphasis. Teachers could choose from 



several kinds of instructional materials; there. was more emphasis on having 
teachers develop their own lessons which were to emphasize classical, 
psychological problfim solving strategies; and instruction was tied more 
directly to the regular district textbook and standardized testing programs. 
Even with this freedom to tailor TOPS to locally perceived heeds and local 
materials, problem solving was never completely accepted by "all the. teachers, 
but was viewed by some as competing for math instruction time. 

The two approaches led to quite different -implementation results. At Site 
A, approximately 80 junior high mathematics teachers were trained, eventually 
impacting over 300 classes. The school board pledged support to continue 
training of new teachers next year with' local funds and the long range goal is 
to train all middle school teachers. TOPS goals do not particularly match 
those of the present district-admin'istered test. As a result, the TOPS 
program has provided an impetus at Site A to develop, over the- next few years, 
a new series of tests in mathematics. They will emphasize problem solving and 
become part of the district's testing program. » 

At Site B the undertaking was in a sense, more ambitious, in that 
"general i st'.' teachers who taught in elementary schools and who, had a less 
strong mathematics background formed the first group of trainees. This 
emphasis on training the generalist teachers, and elucidating for them the 
more generally accepted strategies of problem solving with a very careful eye 
towards standardized test performance, may make the- goals at Site B more 
appealing to some school administrators. But progress is bound to be slower 
and more difficult* because of the- need to train teachers in the mathematics 
and in the problem solving strategies. These needs are compounded by the 

• 1 1 * t 

* > « > . 

need 'for teachers to develop for themselves" lessons embodying the mathematics 
arjd the strategies. A major problem for teachers who want to teach problem 



solvit daily as an adjunct to the text given the absence of gooy'curricular , 
materials are available for teaching these particular strategics. 
Summary of Effects. There werte two major effects^. They wer/: , ' • 

I. At both sites, participants 'viewed TOPS favorably a/d commented on its 
positive impact on students' mathematics and linking' srkil Is. 
II. At both sites, TOPS students had better test scores. £han Non-TOPS 
students. But in spite of consistently higher scores, the differences were 
statistically significant at Site A but not 'at Site B; the scores do suggest 
TOPS fosters' problem solving development. 
Both findinas are worth amplifying: 

I. Overall, TOPS was considered successful by participants. All 
participants viewed the TOPS project favorably. The majority of teachers 
thought that their ability to teach TOPS had improved since last year and also 
indicated they were capable of continuing to teach TOPS next year and would be 
likely ,to do so: The majority of. teachers indicated that TOPS tfad a 
beneficial effect on their students and Coordinators favorabTy reviewed 'the 
majority of lessons, the impact on students and teacher's mastery of those 
lessons. 

According to participants, the. TOPS approach, designed and executed by the 
CEMREL staff, -was a challenging, substantive, and productive way to teach 
.problem solving. "As an alternative to current texts and materials, TOPS was 
appreciated. by some, if not all, of the teachers^ For teachers who wanted to 

r - 

teach problem solving but couldn't find an entree, TOPS worked, and all 
participants .regardless of problem solving expertise commented favorably on 
TOPS quality^ and on the ability of CEMREL' s TOPS staff and their local staff 
to introduce the teaching or problem solving as well as they did. Particularly 



appreciated was the role defined by TOPS for. the Coordinators. By casting 
them in the role of consultants and collaborators (as opposed to supervisors), 
the project designers set the stage for interactive professional development' 
that teachers found helpful. 

II. Student Achievement . At Site A, TOPS classes had significantly 
higher scores than Non-TOPS classes on the MANS Tests, a special/series of 
problem solving tests. The adjusted mean scores were 128 (for six seventh 
grade "fops classes) versus 110 (for six seventh grade Non-TOPS ,c1 asses) . 
These TOPS classes also had-higher scores, by about 1 0%, on the Concepts and 

.Applications test of the California Achievement Tests, although this 
difference did not reach statistical significance. There are two factors 
which make these findings especially noteworthy. First ; 7 when problem solving 
is measured by tests which do not test directly what. was taught (i.e., are new 
to students, as was the case in Site A), significant -differences between 
curriculum groups are not_ usually found. Second, most of the TOPS students 
had only one year of TOPS instruction; one can expect even stronger results if 

'students study TOPS throughout middle school. 

At Site B, TQPS students had higher scores than Non-TOPS students, but the 
differences were not statistically significant. On the California Achievement 
Tests, Concepts and Applications, for nine fourth grade TOPS classes the 
adjusted, mean scale score was about 431 versus about 425 for eight Non-TOPS 
classes. The mean score for a scattered group of fifth grade TOPS students w 

."completing their second year of TOPS was about 469 versus about 461 for a 
comparable Non-TOPS group. * 
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Differences between the tv*5 sites in teachers' background and program 
implementation are instructive- We conclude that districts which train 
"specialist" teachers, who teach mathematics to several classes each day, will 
get a better return on thei'r training investment. Since these- teachers need- : 
to spend less time learning the mathematical processes embodied in TOPS, more 
time is available for observing and practicing TOPS teaching techniques. On 
the other hand, a district whicbl.decides to adopt TOPS in order to give less 
.mathematically able teachers an introduction to mathematical processes^? 
problem solving, as Site B did, will need more time for training and that 
training will be a more difficult undertaking. 

Districts also need to make 'an explicit commitment to the teachi 
problem solving. Teachers need assurances that their attempts will be 
supported and sustained, and that evaluations of TOPS impact on students will 
bear some relationship to instruction. Evaluation attempts also need to 
recognize that ^he implement aton process and its effects on students may 
happen slowly. . ' - . . . 

Given those conditions, TOPS can make a positive contribution to districts 
interested in improving the mathematical problem solving ^W34 s tjej$ of their 
students. , ' ) 
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/ . INTRODUCTION 

TOPS,' a program designed to teach prpblem solving in mathematics, was 

developed by CEMREL staff and implemented by that staff in conjunction with 

administrators and teachers at two pi lcrt sites, Si.te A' and Site B, from fall, 

1980 to Spring, 1982. Both -sites are large urban school districts." TOPS 

goals were-to train teachers to become capable of teaching effectively for 

problem solving there by improving students' problem solving skills. Special- 

emphasis was placed on improving students' facility with applications and with 

i »• 
higher level cognitive skills, rather than computation. 

Need for the Program * » 

4 ft 

Impetus for the project came from several sources, among them NCTM's 

statement that problem solving in mathematics should be an important part of 

mathematics instruction. A similar statement by the National Council of 

» 

Supervisors of Mathematics served to underscore the importance of problem 
solving in . the curriculum. Both groups supported teaching problem solving at ^ 
a time when students' standardized test scores, especially in mathematics, 
were lower than they had-been, in a decade; ■ As a result, school districts 
began looking for curriculum materials and instructional techniques that 
emphasized prpblem solving. ' 

TOPS: A Proposed Solution to Districts' Need for a Problem Solving Curriculum 

CEMREL 's CSMP staff£had developed, over a number of years, a mathematics 

C • 
curriculum which stressed a problem solving approach to computation, number 

relations, and other mathematical topics. 

■/ 



TOPS was designed from components of the program with the goal of teaching , 
problem solving to elementary school students one day a week. Both. CSMP and 
TOPS are based on the assumption that problem solving li'es a't the heart of 
mathematics, and should be taught as part of students' mathematics ■ t 
instruction. To achieve .this goal, TOPS materials and strategies,, like 'CSMP, 
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were, developed from ideas found in advanced mathematics and were designed to • 

i % • " * 

supplement students' standard mathematics curriculum. 

h % 1 v » ' 

TOPS goal of draining teachers in problem solving was based on the • 

assumption that teachers couk! be taught how to appy problem solving • 

techniques in their own classrooms. According-ly, training was targeted to the 

TOPS staff helping teachers lear\and apply problem solving strategies. 

First, in workshops, and then through Coordinator classroom visits as the 

teacher taught TOPS in lieu of the iextbook one day a week. , It was * 

anticipated that the combination of training and practice, monitored by 

district level coordinators, would enable the teachers" to increase their . 

districts' ability to teach problem solving to more and more students each 

year, apd as a result create long range- improvement in problem solving in each 

district. 

As TOPS was conceived, both goals were to be met^by using techniques, 

materials, and strategies based on a specific. point of view which was 

\ \ K 

operational ized via five major sets of activities: the Language of Arrows, 

the String Game, the Mini -computer, Hand Calculator* and Detective Stories. 

They combine mathematical content, like number relations, with basic cognitive 

processes, like discovering a pattern which describes the relations. Since 

presenting the' content calls for a specific style\or teaching, and- TOPS is 

based on a point of view which is not widely known; teacher training is 

■ • i 

necessary. Conceptualization not computation lies at the core of the program, 

• . i 
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which means a TOPS classroom may not resemble the typical classroom in which 

» - • 

teachers learned or previously taught mathematics. In the beginning, teachers 
need consultations in planning and teaching lessons to foster their 
development of students 1 problem s'ol ving. 
Evaluating TOPS : 

In evaluating TOPS, the dual goals of training teachers and improvjrig 
students 1 problem solving served as focal points for the evaluation ^ 
questions. . The'two key issues, which shaped the evaluation, were: 

o Was TOPS, as envisioned by its'developers, 

successfully implemented at each* site? 
o Oid TOPS participation affect students 1 

mathematical problem solving ability? 

A third question: 

o .What is the likelihood of a program like TOPS 
. succeeding at other sites? 
i\s also addressed. To answer the three qgestions, v data were gathered from 

several sources: ♦ , 

- <? - 

o ' Interviews with coordinators, principals and 

teachers. / 4 

o t Questionnaires distributed to all project 
. '* teachers at the end of each project year, 
o/ . Classroom observation data, 
o Tdst data, 

and analyzed 'to produce answers to the questions. 

TOPS. implementation did not proceed as> originally planned at both sites. 

\ 



At Site A implementation was' continued as planned during Year II,' but at Site 
B there were changes in program structure and focus, in personnel, and in 
continuation/expansion prospects. Those changes rendered the two sites 
different, hence the data from each site are best presented separately. .(For 
information'- about Year I of TOPS see Project TOPS ; First Year .Evaluation 
Report . Martin Herbert and Gail Marshall. CEMREL, Inc., St.. Louis, Mo., 1981.) 
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SITE BACKGROUND INFORMATION 
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This?seetit>n will briefly describe the commonalities at the two sites and 

< * » 

.the remainder of the report will describe differences in the implementation 
and impact of. TOPS.^ first at Site A, and then at Site B. 
The Sites . ■. 

Site A is a large mid-western urban school district. A major manufacturing 
center,, fifs urban, area typifies the economic, social, and educational profile 
of many large ./American cities. A downtown decaying and rebuilding, white 
. flight, and a high concentration of poor and unemployed mark the problems 
faced by the v school administrators. Site B is a middle sized urban school 
district where both heavy and light industry provide. a more diversified 
economy. However, like Site A, its problems are typical of cities its size. 
Site Events 

The contract for TOPS began October 1, "1980. At each site a district level 
-Coordinator was chosen, by site administrators and a Demonstration Teacher was 
chosen to work with the Coordinatbr. t Following the selection of these 
district level staff, who were to be chose'n because of their mathematics 
training and their administrative experience, schools and teachers were 
chosen. Jn some pases, schools' and teachers were selected because of low test 
scores in mathematics. In some cases, teachers volunteered for the program- 
..and their classes did not necessarily follow that pattern. In still other 
leases, the" teachers chosen were those who needed special help in learning or 
V; : teaching mathematics, as perceived by a principal or other administrator. 
This happened more frequently/at Site B. 
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Once the staff \nd teachers were chosen, the first series of workshops at 
the sites'was scheduled for mid- November,- 1980. At Site B, the workshop was . 
held as planned (November 12rl4), almost three months after school began, a 
beginning which was later- than originally anticipated. But at -Site A-, 
district level decisions "led to the postponement of both, the November workshop 
and one rescheduledfor December. The workshop was finally held January 20-22, 
1981. Since formal TOPS teaching began the Monday immediately following the 

* workshops, Site A TOlPS instruction began halfway through the school year. 

» There were four major differences between the two sites which had . 

* implications for the day-to-day conduct of the project and for the future of 
the project. These differences, which will be discussed in turn, were: 

o Staff experience and training . 

o Schedules for teaching TOPS 

o Definition and operational izing of problem solving. 

o Plans for continuation and expansion- 

Staff Experience) and Training. Analysis of the questionnaires completed by 
staff at both sites (Appendix A) showed that the Coordinator and Demonstration 
Teacher at Site A had more formal mathematics training and more experience 
teaching and supervising mathematics clashes than the Site B staff. 

Teachers at Site A had more formal mathematics training; more of them 7i ad 
a major, in mathematigs or a concentration of mathematics courses as 
undergraduates. Site A teachers often taught TOPS at several grade levels and 
they taught only Math to those classes. Site* B teachers were more likely to 
teach Math and other subjects, and most often taught elementary classes. ** 
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Thus, Site A teachers could" be considered "specialist" teachers, whose 

primary responsibility was to teach mathematics. By virtue of their training 

and their assignments in schools, they were- specializing in mathematics 

instruction. Most Site B teachers could be regarded as "general ists", 

v 

teachers trainee! to, teach^a wide variety of subjects and assigned to a ^ 
particular grade level. > 1 

A second difference in staffing also occurred at Site B. On the 
recommendation of the Federal Project Officer, the'two Demonstration Teachers, 
each of whom had worked half-time on the project, were replaced by a full-time 
Demonstration Teacher at the end of Year I. This change meant that the 
Coordinator had to acquaint the new Demonstration Teacher with the project 
while planning for the-second year in-service, and had to re-schedule 
classroom visits, since the previous year's .schedule (predicated on two 
part-time staff members) was no longer workable. The Demonstration Teacher had 
not participated in Year I in-service meetings and seminars with CEMREL staff, 
and was therefore not aware of the point of view and operational strategies 
which had been implemented at Site B during Year I. 
Scheduling. A second mandated change from the Federal Project Officer 
affected the teaching of TOPS it Site B. Concerned about the match between 
TOPS goals and the district goals, a match which had been questioned by the 
Coordinator and several teachers, the Federal Project Officer directed TOPS 
teachers to teach problem solving for a few minutes each day in conjunction 
with topics and -problems covered in the local textbook. At Site A, teachers 

v , 

continued to teach TOPS once a, week as originally envisioned by TOPS planners, 

but at Site B, a new schedule had to be worked out by the' teachers, and new 

ways of integrating TOPS had to be devised. In fact, this mandate to teach 

■ * 



TOPS every day crystallized a latent problem at .Site B. Several teachers, not 
strong in mathematics to begin with and just beginning to gain mastery of the 
TOPS materials and concepts,, were unable to integrate*, them into their local 
textbook program.' for example, some teachers were able to see the 

applicability of "the Mini -computer to place value lessons, or the String Game 

*** 

io factoring, but other teachers were at a loss as to how to use TOPS 
materials ar*d strategies as adjuncts to the text." 1 . . 
Definition and teaching of problem solving. At Site A, teachers continued 
Using the types of. TOPS lessons which they had practiced the year before, as 
^well as- refining their u^e. of' the problem solving strategies embddied in'them. 
At Site B,' the mandate by the Federal Project Officer resulted in a'' wide range 
of implementation -strategies'; 'The "net result was a melange of classroom 
instruction, at Site B, some of which could 'be called problem solving. using a 
TOPS, focus, some of which was a more generic form of problem solving, and some 
of which had no discernable problem solving orientation. 

The difference's in teachers' intuitive definitions of problem solving and 
in their willingness to 'accept TOPS materials ahd concepts, appeared to be _ 
related to their Mathematics training. Teachers who had had a concentration 
in mathematics and/or taught Math to several grades each day (as at Site A ) 
were more likely to maintain fidelity to TOPS lessons and. to yse TOPS . 
materials and strategies to focus on the problem solving aspect of the 
material^s^Their mathematics prepared them to understand and maximize the 
lessons. For teachers with weaker mathematics backgrounds (typically at Site B 
), T0Ps)was usually nothing more than an introduction to problem-solving 
strategies , and .to non-computational ly oriented mathematics. 
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Continuation and Expansion. In August, 1981 while Site A was in a position to 
build on and improve the first year's work, Site B, for all intents and 

• • • 

purposes^ had to begin anew. Since the changes mandated by the Federal 
Project Officer had introduced a new staff, a new teaching schedule, ,and a new 
problem solving focus, ^some teachers were conf used about'- the actual purpose of 

•TOPS instruction. Hence they were not effective advocates of the program. 
More detrimental to continuation and expansion efforts however, .was the fact 
that a major portion of the introductory workshop sessions at Site B in 
1981-1982 had to be devoted to explaining the changes to 4e original group of 
TOPS tealchers. Schedule changes and their concurrent reporting requirements 
also commanded time, and energy. This meant the Coordinator had no /Time to 
think about trainyig a second cadre of teachers or helping TOPS teachers strain 
other teachers in their buildings. Instead, four replacement teachers were 
phased into the project by participation in Year II activities. While the new' 
staff brought insights and,\skills tffat contributed to the project, working 
relations with project teachers in some cases had to be built from scratch. 
' The Site A Coordinator and Demonstration Teacher, on the other hand, were 
able to spend the first few weeks of Year II reviewing T0P» teach ers^wofk "and 
introducing a. second cadre of teachers to TOPS. Later in the year, they were 
able to monitor both groups, and introduce a third group of teachers to TOPS. 

. This expansion resulted from the continuity of efforts which characterized 
Year II at Site A. 

The differences between the two sites affected TOPS implementation and 
impact. .Both issues will be discussed in. the next sec.tion. 
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TOPS IMPLEMENTATION: SITE A 

Data in this section were gathered by means of questionnaires mailed to 
TOPS teachers in Spring, 1982. Twenty eight teachers returned completed 
questionnaires : twelve were teachers who had been in the project last year 
(69% of the veteran 'teachers) and sixteen joined the project during Year II 
.»(34% of the the. new teachers). Thus the data reported below represent some; • 
but not all, of the teachers currently teaching TOPS at Site A. .Data from 
veteran and new .teachers were aggregated in all cases except where the 
responses of veUegan teachers differed from those of the new teachers. 



l r. Background and Experience . The Coordinator , had advanced degrees in 
a Mathematics^ atrd 'extensive experience as a Mathematics supervisor at the middle 

• school leye>. The Demonstration Teather had experience with CSMP as well as . 
s Mathematics supervisory experience. In the ta\les which follow, data 

describing teachers' background and experience are presented. 

Teachers 1 $£ckground and Experience ■ ' 

Number of. 'Years Teaching * 
1st year^ 0% 

2-5 years 18% ^ 

6-10 years 35% 

10-20 y£ars 31% \ 

20 plus years 1 6% 

> 

Highest Academic* Status Attained ' 

Bachelor'.? plus' 3151, Master's 48%, Master's plus 2151 

College Preparation ' ♦ 

• Elementary education Elementary education Another major with Concentration 
major with no math with 1 or 2 math some math courses in mathematics 

methods courses methods courses 

0% 30% 30% 40% 



1 i , 



ERIC 



11 



1/The majority of the teachers tend to be experienced teachers, have 

received advanced degrees, and have taken mathematics courses in'college. As, 

/ 

a group, the new teachers .had more experience, a better background fn 
mathematics, and were more likely to have volunteered for TOPS (as opppsed to 

!' 

being selected) than the veteran teachers. 

2. Teaching Schedule . Twenty-five Jf the twenty-eight teachers taught math at 
least four times a day, usually in at^least two different grade levels. All 
were middle school teachers (grades five to eight) and all but one of the 
teachers taught TOPS one full math period, per week-. 

3. TOPS Teacher Training . Before beginning theiw TOPS teaching, the teachers 

attended a week' long workshop. Conducted by CEMREL'S TOPS staff in 

conjunction with the loca'l Coordinator "and Demonstration Teacher, the workshop 

was designed to acquaint them with tOPS/materi als and strategies. Their 

s 

reactions to this workshop, and to the day-to-day Coordinator activities which 

' V 

followed the workshop, are summarized below from questionnaire data. (A 

» 

listing of teachers' evaluations is shown in Appendix A.) - 



Distribution of Workshop Topics and Ratings of Effectiveness 



Overview of the program's 
philosophy and goals 



Discussion/presentation of 
math content by trainer 



Practice by participants 
teaching various lessons 



Average 
'Percent 
of time 




Ratings 




*132 


69* 


■ ts% 


\ — 6% 




effective 


somewhat 
effective 


not 
effective 




852 


122 


3% 


242^ 

« ** 

: 472 


. effective 
912 


somewhat 
effective 

9* 


non- 
effective 

1 OS' 


s - 


effective 


somewhat 
effective 


not 
. effective 


162 


662 


172 


17% 




effect i ve 


somewhat 
effective 


not 
effective 
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Strengths and Weaknesses of the Workshops. Several teachers agreed that too 
much material was covered in too short a time, and they would have liked to 
have learned a small segment, practiced it, and then met again for another 
segment, 

On the positive side, 'teachers liked the demonstrations of lessons and the 
opportunity for participants to practice, .especially with students present. 

Strengths and Weaknesses of Coordinator Activities. Only one negative aspect 
of the Coordinators' activities was mentioned: a teacher stated that the 
observations. created tension. All other teacher comments were positive and 
'teachers-cited the knowledgeability and helpfulness, the offering of ideas, 
arid the demonstration' of new and different ways to deal| with- students as 
strong features of the training! 



Rating Coordinator Activities. 

"If a Coordinator were available next year, what activities would you -like to 
see emphasized?" 



Conducting demonstration classes 
Reviewing mathematical content 
Planning future lessons ? 
Discussing general problem solving strategies 
Critiquing lessons you taught 
Explaining the goals of an individual lesson 
Suggesting classroom management procedures 
Classroom visits in general 



very 


somewhat 


not us'ually 


helpfdl 


helpful 


' helpful 


74% 


20% 


-6% 


445t 


. 38* 


18* 


48* 


szi 


0* • 


58* 


42% 


0* 


55* 


' 45* 


0* ' 


44% 


38% 


18* 


341 


• ,66% 


0* 


55% 


45% . 


0% 



New teachers, who had more mathematics in college, were less likely than 
veteran teachers to r.ate "discussing general problem solving strategies" as 
very helpful (45% vs. 71%) but more likely to rate "planning future lessons" 
as very helpful (62% vs. 33%). 
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4. Teaching TOPS: -distribution of TOPS Lessons 

The major part of the TOPS program was the weekly planned lesson taught by 
the training teacher 'and often observed and critiqued by the Coordinator or 
Demonstration Teacher. The teachers kept logs of what was taught and these 
are summarized beTow. (Because the length of the TOPS school year differed 
from Year I to Year II, the. percentage of lesso.ns devoted to each strand is 
shown.) 



/ 



String Games 
Arrows. 

Mini -computer 
Hand Calculator 
Detective Stories 
Geometry 



Veteran Teachers Teachers 
'This Year Last Year 



26% 
25% 

TT% 



32% 
17% 

m 

77% 
1% 
T% 



Greater uniformity .i n lesson coverage is shown this year-. 



5. Coordinator Ratings of~ TOPS* Lei sons 
Durincj classroom visits, the Coordinator and Demonstration. Teacher rated the 

< 

lesson in'a log. -Six major aspects of the lesson were to be rated, and the 
average rating across^all observed lessons is given below for 'each aspect. 



Category 

Students' Interest 
Students' Understanding 
Teacher Preparation/Understanding 
Teachers' Enthusiasm/Delivery 

•Use of Materials 



Teacher/Student Interaction 



High 
82% 

S3* . 
8T% 
.57% 
Appropriate 

Most 
Students 
Involved 
' 84% 



Ratings 

Average Low 

•15% " 3% 

m • 7% * 

T2~% • 6% 

n% 

Inappropriate -Had Difficulty 

3% 4% . 

Strong Weak , 

Students Students 

Involved Involved 

16% 0% 
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Another way of analyzing the same log data is to calculate average ratings 
across lessons according to the content strand of the lesson. These are shown 
below. 



Lessons 




Ratings 






High 


Medium 


Low 


Arrows 

String Game. 
Mini-Computer 
Hand Calculator 
Detective Stories 


78% 
90* 
53* 
75* 
9l* 


17* 
^9* 
V5% 

m 

- T* 


5% 
T% 
J% 
7% 



Both analyses show that lessons were generally rated positively by 
Coordinators. ' 
6. Teacher Ratings of T0P> Materials 

This year, like 'las/6 year, teachers rated\oPS materials "Low" or "High" 
on five categories. 





String 


Ctnguage of 
. Arrows 


Hand 
'talculator 


M1n1 
Compu 


ter 


Detect 
Storl 


1ve 
es 


He ah 
Rating 


. 1 


Can be used 
affectively 
by a teacher 


Tfigh 
100* 


Low 
OX 


mgn 
100* 


LOW 

ox 


High 
64X* 


Low 
36X 


-RTgH" 
76X 


Low 
24X 


High 
-88X 


Low 
12X 


High" 
8S.6X 


Low 
14.4X 


Promoting student 
growth in problem* 
solving skills 


93*. 


7* 


100* 


ox 


SOX 


SOX 


"90X 


10X 


*.89X 


nx 


84.4X 

- ' 


15.6X1 

. 1 


Promoting positive* 
student attitude 
toward problem- 
vsolving 


100t 


0* 


93* 


7X • 


67X 


33X 


90X . 


10X 


89X 


nx - 


87.8X 


12.2X 

; i 


Student enjoyment 


94% 


6* 


63* 


37X 


78X 


22X 


89X 


m 


72% 


18X 


80.2X 


19.8X 


Overall benefit to 
students compared 
to the math period 
it replaced 


63* 


37* 


85* 


14X 


67X 


33X 


85X 


1SX 


83X 


17X 


76.8X 


23.25 


Mean rating 


90* 


10* 


88* 


12X 


65X 


35X ' 


86X 


14X 


84X 


16X 


82.6X 


17.4* 




2. 



Each of the' lesson strands received 84-90% high ratings except Hand 
Calculators which had 65%. Similarly each Category received consistently high 
ratings except "Overall benefit to students compared to the math period it 
replaced." which still received an average of 77% high responses. Generally, 
ratings were higher this year than last year. . . 

t 

• > ■ , ■ . t 

7. Teachers' Assessments of TOPS' Features 

c 

The statements below were generated from negative comments one or more 
teachers made during. observations and interviews conducted in Spring, 1981. 
(Responses, do nofal ways' total 100% because of the pattern of teacher 
responses.) * . t " x 



Strongly Strongly 
Agree Agree Disagree Disagree 



Teaching TOPS takes too much • 
preparation time. 13% 63% 19% 

TOPS^ideas can be applied in . 

situations outside 1 TOPS * t . 
- classrooms. ■ • '. 42% 55% * 



TOPS does not conflict witffo 
our school 's math goals.- 38% . 59% 3% 
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TOPS does not contain enough " 

'drill. ' _J% . 22% 61% ' 11% 

Tests should be included as'* 

< "part of the TOPS materials." ^_8% _49% ■ J0% 3% 

TOPS is not useful in any 
other context." t \ - m 63% 29% , 

TOPS goals are clear.' 29% ' 53% 15% 

.- t . ' 

TOPS does not help me learn 

more about my students , 3% 68% , 29% 

» . ; 

The game aspect of. TOPS 
lessens its value. * 63% 37% 

Students Who have had TOPS . 
this year should continue 

with TOPS next year. JQ% J50% 



In general, these ratings seem to reflect positive teacher attitudes 
toward TOPS. .For most questions, answers were more positive thisiyear than 
last year. Negative comments received from individual teachers a^e not widely 
shared except for the statements referring to TOPS* not containing Enough drill 
and the need for test materials as part of TOPS where 25% and 57% Respectively 
agreed with these statements. 



8. TOPS Teachers' Views on Teaching TOPS 
Having taught TOPS for two years, veteran teachers were in a position to 

compare their first year of teaching TOPS with their second year. Generally, 

w 

they noted three changes, 1) better command of the material which l,eadlto the 

* • >• I 

ability to plan and organize their TOPS teaching, 2) . greater ability toj 

coordinate the material with' the*Yegul ar currrculum, 3) greater ability.|to 

. positively affect- student - achievement. • \l 

Veteran and new teachers. were asked what changes in their teaching tjiey 

/ * \ i 

would make as a result of JOPS teaching. Both groups stressed questionifttf 
students more, asking students to explain answers,. al lowing students to make 
discoveries, and giving students time to think through a problem on their own. 

Asked to discuss persistent problems they had implementing T0ps..t'his year, 
one teacher cited a problem-of sequenting lessons, and two cited difficulties 
holding and keeping students 1 attention. Another teacher reported having 
difficulty allocating time for both TOPS activities and other mathematics 
activities in the districts curriculum. 

* 

9. Teachers 1 V.i ews on TOPS Continuation 

As a result of their TOPS experience, all teachers in both groups, veteran 
teachers "and new teachers, said they would be able to teach TOPS next year; 
and all but one teacher in .the new group would choose to teach TOPS next y?ar. 
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When they responded' to the Question "What changes would you' make 'in TOPS . 

next year?", teachers' .answers indicated they understood the purpose of -TOPS 

lessons, and agreed with the TOPS approach, for example, teachers said th.ey 

wouHd ask more questions, and use TOPS materials and techniques more 

> • 
frequently in their regular math classes * They also said they would be more 

flexible about what days they taught TOPS, perhaps teaching TOPS lessons every 

,day some weeks and a full math period one day a,.week at other times. Teachers 

*" ' " ' • 

also said they would prefer to initiate Coordinator visits instead of having 

regular visits scheduled by the Coordinator. . • , • ' 

* • - ' • _~ . *- " 4 ' 

Several teachers said theywouid need no help' next year, several would 

like workshops scheduled so they could share ideas or receive additional 

materials and lessons, and others would like only replacement materials and 

occasional consultations with Coordinators. 

An important aspect of TOPS* continuation is teachers 1 ability to do an 

r " 

effective job with it. About half of the veteran teachers said that at least 
'75% of the teachers of their grade '1 evel (s) could do an effective job with 
• TOPS, and about a quarter said'between 50% and 75% could do so. An but one 
teacher, a new ^eacher, said they would like to see TOPS used in other classes 
in their schools. , . 

According to teachers, among- the problems that new-tb-TOPS teachers would 
face are becoming familiar with and securing TOPS materials, learning to 
organize and plan with materials, learning how to handle the question-answer 
dynamics in front of a class, ind understanding the goals of- the program. 
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One aspect of continuation of TOPS is teachers' perception of what is needed 
if they are to continue teaching TOPS. Three separate issues were mentioned: 

o' There is a need for defining and explaining lesson goals and stating 

* how lessons fit into a sequence of previous lessons. 

o There 1s also a need to design Hand Calculator activities for nfore 
student understanding and enjoyment of the lessons, rather than just 

* • "punching buttons". • " 

o There .is concern about what wiVl happen to- students who have had TOPS 
the previous year.. Teachers asked what. follow-up and sequencing will 
be provided for those students. ' •" • 

- . • i% 

Another aspect of continuation 4 will be teachers' reliance on otheV 1 
teachers in their buildings for help with TOPS /lessons. There is evidence 
that teachers are helping one another, but in a limited way. A few teachers 
said they offered suggestions on how to teach specific lessons, answered 
teachers' questions, and shared techniques that they themselves had devised or 
learned. This is a low level of teacher-to-teacher support which will not be ; 
sufficient in and of itself to expand the program and may not" even be enough 
to sustain the participation of new teachers who do not yet understand how to 
teach TOPS. • ' , .. 

10. Expansion Prospects ' 
V ~~ 

One of th^Jistrict's* goals for TOPS was the development of a* cadre of 
teachers who could act as change agents and expand the teaching of problem 
solving. This cadre of teachers played only aj)imited role (and much less 
than originally planned) in the process of expanding to new teachers. 
Nevertheless, the Coordinator and Demonstration Teacher have far exceeded 
expectations in the training of new teachers. One group of teachers has been 
trained, observed, and at present, is almost fully autonomous. They can call 
upon the Coordinator when needed but they are viewed as being able to function 



on their own. Having succe'ssf ully, launched one group of teachers, the site t 
staff turned their attention to training two more groups and currently those i 
groups have begun teaching TOPS under the staff guidance. About 10,000 * 
students were receiving TOPS instruction by May*, 1982, or about a quarter of 
the district^ students tn those grade levels. . „ 

The coordinating staff have also written (lessons, prepared a teacher 
training program, and written a handbook describing training. 

Based on these successful steps toward expansion, the district is 
currently working* on ways to institutionalize TOPS in the district. In spite 
of critical budget problem^and- a teacher strike, the School Board has . 
authorized continued support of TOPS, entirely from local funds, after the end 
of federally funded TOPS. • 

11. Overall Reaction 

In an open-ended question on the questionnaire, teachers were asked to - 
give an overall evaluation of TOPS. Their evaluations of TOPS were uniform^, 
positive. They stated that TOPS participation had a beneficial effect on 
their teaching and a'positive. impact on their students. 

In summary, teachers' ratings of the workshops, the lessons, and TOPS 
features were favorable. Teachers^eem to have a practical and conceptual 
command over TOPS, and express an interest in continuing TOPS. Expansion is 
taking pldce more rapidly than planned, and the district is commited to^ 
continuing and expanding TOPS with local money. , j 
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TOPS IMPACT: SITE A * ^ 

This section is divided into two parts. The first part will present 
teachers' rating of TOPS impact regarding: ^ 

■ o " TOPS effectiveness in improving students' thinking, skills 

o TOPS effectiveness in improving students' ability to do specific 
mathematical tasks • - 

o TOPS value of teaching strategies 
-The second part, will present test data from: * ^ 

o • The MANS Test 

s 

o Standardized achievement tests A 
1. Teachers' Ratings of TOPS Impact 

TOPS effectiveness in improving. students' thinking . 

Much ♦ Some No 

\ Improvement Improvement Improvement 

Analyzing Situations 52% 48% 

Reasoning Logically 49% > 5T% 

Evaluating Possible Answers - 52% 

Reflecting Before Responding 48% 52% ■ 

Producing Multiple Answers 3£% P% 

These_ratings were somewhat lower than last year. 

TOPS' effectiveness in improving students' ability to do specific mathematical 

tasks. . , * 

' Much 'Some , No 

' Improvement Improvefnent Improvement 
Place. Value • ' 20%' 21% ' 7% 

Fractions 
Estimation 
Word Problems 
Decimals 

Mental Arithmetic - ■ - 
Ratings are slightly- higher than last year. 



4T% 


. T7% 




47% 


37% 


16% 


m 


72% 






4U% 




77% 
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Value of TORS teaching strategies 



Less pressure 

(no "wrong 11 answers, everyone 
v can contribute)* 

Content 
(good problem situations, 
lesson plans) 

Student responses 
(explaining "how. to" and "why", 
many possible answers) 

Hands-on material [ , - 
(Mini -computer, hand calculator, 
strings) 

Game atmosphere . \ 
(no paper pencil, team piay) 

Teacher questioning 
(thought provoking, open-ended, 
follow up to student responses) 



Helps 
A Lot 



78% 
72% 

V 

88% 

85* 
70% 

77% 



Helps 
A Little 



22% 



28% 

12% 

10% 
25% 

23% 



Doesn't 
Help 



5% 
5% 



Again this year„ "Hands-on materials" and Student responses" received 
'the highest ratings. Overall, ratings were similar to last year. 
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2. Test Data 



The MANS Test Data 



To assess the effect of TOPS instruction on participating students, a 
version of the MANS Tests (Mathematics Applied to Novel Situations) was , "' r 
constructed specifically for TOPS and was administered to TOPS and non-TOPS 
classes. 

The. MANS Tests. The MANS Tests are short test scales originally designed to. 
-assess what are thought to be some of the underlying thinking skills of CSMP, 
the ^curriculum on which TOPS was based. They have been used for several years 

".in the evaluation of CSMP in second through sixth grade. Since the thinking 
skills fostered by TOPS are similar to CSMP, an adaptation of the tests for 

. TOPS evaluation seemed appropriate. 

The MANS Tests used in this evaluation consisted of 20 separate tests, 
grouped into four categories: Estimation and Mental Arithmetic; Fractions and 
Decimals; Number Patterns and Relationships; and Production of Multiple 
Answers. (A description of the tests and their administration is given in 
Appendix B.) It, is important to note that these tests do not refer to any TOPS 

- lessons orcontatn any of the specific terminology of ^TOP-S;- indeed most were 
designed especially to present mathematical tasks which the students had not 
encountered previously. A specially trained tester followed a standardized 
script "in introducing, one-at-a-time, the various tests. 
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The Sample. Twelve classes of seventh grade students were tested (6 TOPS and 
6 non-TOPS classes). The seventh grade^as thought to be the most appropriate 
grade level for.comparing TOPS and Non-TOPS students' performance. One class 
from each of six veteran TOPS teachers was tested. One class from each of six 
comparison Non-TOPS teachers was also tested. 

Selection of classes' to be tested was made by the district staff. 
Non-TOPS classes were selected so that, in each case, classes were comparable 
in ability. to TOPS classes and teachers' ability was judged comparable to the 
corresponding TOPS teachers. 

For several "reasons, TOPS classes could not "be kept together from 
year-to-year. Hence the original evaluation plan, which called for the 
testing of TOPS students with two years of TOPS instruction, could not' be 
followed. Five of the six classes were composed almost entirely of 
new-to-TOPS students. 

Method of Analysis. For each class, a mean score was calculated for each MANS 
test. A class mean score was also calculated for each class on two' 
independent measures: a. the district administered Reading Comprehension 
subtest of 'the California Achievement Test. b. a Figural Reasoning test, 
consisting of items in which stude'nts had to decide which one of four give v n 
figures was different from the other three. 'This was considered to be measure 
reasoning of a very different kind from that of TOPS or any other math 
program/'lThe mean scores for this neatest. were 7.9 for TOPS classes and 8.2 
for Non-TOPS classes; The mean raw sfore across TOPS classes on the reading 
test was 19.0; for. Non-TOPS classes it was 20.2.) 
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Thus, the classes were reasonably well matched in ability. An Analysis 
of Covariance procedure on class means was then used so that mean scores on 
the MANS tests .were adjusted to take into account differences in ability among 
the classes as measured by reading and figural reasoning scores. (Appendix C 
presents class means for TOPS and non-TOPS classes). 

Results. Table 1 shows adjusted mean scores for TOPS and non-TOPS classes. 
The tests have been grouped into categories of mathematical processes. Thj 
p-value of the comparison is also given, i.e., .tne probability that a 
difference that large between the two groups coul^. have occurred by chance if 
there were truly " no differences" between the TOP^ and non-TOPS classes. A- 
p-value of .05 or less is often designated as statistically "significant" 
' (i.e., would happen by chance only.once in twenty times). 

< • TABLE 1 
SUMMARY DATA, MANS TESTS 
SITE A 

ADJUSTED MEAN SCORES- 
MANS CATEGORIES, TOPS ANO NON-TOPS 
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TOPS 
Classes 


Non-TOPS 
Classes 


P-Value 


Estimation and 
* Mental Arithmetic 

Includes two types of scales: 

, Estimation requires* the rapid 
deriving of approximate 
answers to problems, and Mental 
Arithmetic' requires 1 the exact . 
answers to calculations amenable 
to non-algorithmic solution. x 


35.6 


31.5 


.20 


Fractions and 

Decimals 4 

Requires computation with 
fractions and decimals in a 
variety of novel application^ 


28.1 


23.6^ 


.15 


Representations and 

Patterns of Numbers 

Requires finding or applying 
a given pattern 1n sets of 
numbers. 


'39.0 


33.1 


.02 


Elucidation 

Requires producing many 
correct answers to a given, 
problem. 


25.5 


22.1 


.15 


Tot?1 MANS 


128.3 


118.4 


.05 



'Based on the t-test" from the Analysis of Covariance with 8 degrees of 
freedom. 

2 A description of the tests in- each category is»given in Appendix B. 

25 
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The table shows that TOPS classes had significantly higher Total MANS' , ? 

scores' than the Non-TOPS classes (using the traditional .05' level of 

: * ' \ I 

significance). They also had significantly higher scores in one category, 

Representations and Number Patterns . In the other three categories, although 

, .05-sigpif icance was npt obtained, the p-values of, at most, .20 are very 

'- ' suggestive (i.e., there' is less than a one-in-five probability th£t these 

differences c^uld have happened by chance). 

Graphs of Class Means. Data from Table 1 are illustrated by- the graph of 
Total MANS scores shown on the next page. Each class is plotted on the -graph 
according to MANS score and ability score (where ability scores is a composite _ 
of reading, and figural reasoning). An X is used for TOPS classes, a dot ( ) 
for non-TOPS classes. 

a, 

A regression line has been drawn on the graph; this line represents the 
best linear prediction of MANS score from class ability §cores. Classes which 
are represented above the line are performing better than expected based on 
their .reading scores; classes below the line are performing worse than 
expected* 

OAfM or CUSS ttNtS 
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Coi»ptfU Atfllty Scort 



Standardized Achievement Tests 
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In addition to showing problem solving ability on s tests designed 
specifically to measure problem solving, differences between TOPS and non-TOPS 
classes were expected on the -standardized achievement tests\administered by 
the district each year. Therefore, achievement test scores (\jie California' 
Achievement Test, administered Spring, 1982) for the seventh grade TOPS and 
non-TOPS classes which participated in the MANS testing were compared. . 

1 ' t 

Tor the same groups of.seventh grade classes, six TOPS classes and six 

|, 

non-TOPS classes, comparisons were made on the math subtests of the district 
administered California Achievement Test'.' An Analysis of Covariance was again 
performed -on the clas's means, this time using only CAT Reading Comprehension 
as the covariate. Mean reading jeores will differ somewhat from the MANS data 
because the exact composition of students present for testing differed from 
day-to-day. The mean reading score'' across TOPS classes was 18.7 and across 
Non-TOPS classes was 18.2. 

Results. Table 2 shows the adjusted mean score for TOPS and non-TOPS 
classes. The p-value of the comparisons is also shown. 

. •» \ • 

TABLE I \ 

SUMMARY DATA, CALIFORNIA ACHIEVEMENT TESTS, SITE A 
ADJUSTED CLASS MEANS 





TOPS \ 
* CI asses 


fton-TOPS 
Classes 


p-Value' 


Mathematics > 
Computation 


19.3 , 
• 


18.6 




Mathematics * 
Concepts and 
Applications 


^ 21.3 < 


19.4 


.20 ■ 


* fotaf 

Mathematics 


40.5 


: — ' : >' ; i 
, 3sil 


.35 



freedom. Where the p-value 1s not less than .5 (a one-1n-two chance) 
a dash (-) appears 1n the table. 

\ 
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results were not significant at the .05 level. - ^ 

I 

Graphs of Class Means. Graphs of class means, like those of the previous 
section, are 'shown below for the two math" subtests of the CAT. It can be seen 
that CAT Math scores are better predicted from Reading scores than were the 
.MANS scores. Although inspection of the graphs shows TOPS classes do better 
generally, th*patt*krof results is too irregular to support a claim of 
superiority for TOPS classes, i.e., suggestive but not conclusive. * 
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TOPS IMPLEMENTATION: SITE -B 

Data in this section were collected by means of a questionnaire mailed 
to Site B TOPS te.achers at the end of Year II. Seventeen teachers returned 
the questionnaire; four were new to the project this year (two replaced Year I 
( T0PS teachers who left the program) the rest were teachers who had taught TOPS 
during Year I* * * 

1. Background and Experience 

During, Year I, a full-time Coordinator, who had a BacheloV's degree in 
mathematics and^expeVience as a school and central office administrator 
directed the project at Site B. He continued that supervision until the last 
months of Yeac II. Two part-time Demonstration Teachers assisted during Year 
I; one had an advanced cfegree in mathematics and was a high school 
mathematics teacher, the other had experience as a supervisor. In Year II, 
both were replaced by a full time Demonstration Teacher. who had both 
mathematics experience and supervisory /administrative experience. 

In the tables, which follow, data describing the teachers 1 background and 

experience is presented. 

Background and Experience . 

Number of Years Teaching' 

1st year 0% 

2 - 5* years 12% 

6-10 years 29% 

10 - 20 years 47% , 

20 plus years 12% , , 

Highest Academic Status Attained , 

♦ ,* . • • ' 

Bachelor's 12% • Bachelor's Plus 18% Master' s\35% Master's Plus 35% 
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College Preparation ' ; ; ■ • * te 

Elementary education Elementary education Another major' with Concentration 
«major with no math with 1 or 2 math .. 'Concentration in in mathematics 
methods courses methods courses some math "courses 

18% ... ' ! 59% 18% 5% / 

• ' < • 

The majority of TOPS .teachers tend to.be experienced teachers, have 

<•.'•;« i • ' • • ' 
received advanced degrees, and their collage background is typically an 

elementary education major withal or Z math methods courses. 

, 4 *- 

Z\ Teaching^ Schedule — ^ o 

More than half of the seventeen responding teachers taught math to only 
one or two classes per day. Less than halt <40X) taught at junior high 
school; the others taught at elementary schools. 

"Is TOPS instruction usually for?" ' 
A few minutes at a time, 

• * 

f 

but not every day . 30% 

A* few minutes at a time 

each day 10% 
A full math period 

each 'week \ ~ • { 24% 
Other v 36% \ . 

These data illustrate the variety of ^implementation strategies TOPS 
teachers used in year II. It is worth noting that only ten percent' of , 
" teachers reported teaching' TOPS for a few minutes each day as newly mandated 

* '• » , t \ 

half way through the project.- 
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3. TOPS Teacher.' Training 

\. \ 

At the first, series of workshops, 1980 - 1-981 , a general introduction to 

: > . 

problem solving»'as defined by the TOPS staff, was presented and that was 



Strengths and Weaknesses of the Workshops. According to teachers, strengths '/ 
of the workshops were: 
o Exchanging ideas, 

o Learning how to use the materials. .... < 
o Learning a variety of problem-solving techniques. ^ 
o Gaining an understanding of the application of mathematics -theory 
through the TOPS activities. • . 

o More "focused" this year (one teacher) 
and weaknesses of the workshops were: ^ \ 

o Not learning how to do Detective .Stories. 

o Needing a more in-depth introduction, i.e., not being thrown, into . 

teaching the activities so -quickly, 
o Not spending enough time explaining„the materials and their 

relations t& problem solving. 

«. • * 

Strengths and Weaknesses of Coordinator Activities. According to TOPS 
teachers, the strength of the Coordinator activities was the dual role of 
teaching and demonstrating followed by observing and offering suggestions. 
The' weakness cited was that too much time elapsed between one Coordinator 
visit and another. ^ 
Rating' Coordinator Activities « v " 1 

Mf a Coordinator were available next year, what activities would you like * 

to see emphasized? 11 • ' 

. . 11 • 

very somewhat not usually 

helpful helpful^ helpful 

Conducting demonstration classes 94% 6% : 

Reviewing mathematical content 60% 30% 10% 

Planning future lessons 53% 27% a .20% 

Discussing general problem solving strategies , 87% 13% 

Critiquing lessons you'taught 64% 18% 18% 

Explaining the goals of an individual lesion 45% 45% , ' 10% 

Suggesting classroom management procedures^ 33% 6 r 6% 

Classroom visits in general 73% 27% 
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' "followed by activities designed to familiarize teachers with-the-materials- and- 

/ 

p.rotedures for -the five TOPS strands, as well as to help teachers understand 

• # ' \he mathematics /mbodied in the, lessons. ' 

Initially.^ th^is" format was to be repeated at each succeeding workshop. 

• However, by the time the August, 1981 workshop was held, the- substantive 
'Changes in the project, mandated by the project officer, necessitated a 

j 

• refocusjng of ttne workshop. As a result, a significant part 'of the 
workshophad to be devoted to explaining the personnel, schedul*, and 
curriculum changes brought abqut by the mandate. Therefore, the focus of the 
August, 1981 workshop and subsequent workshops shifted from practice and 

- growing expertise with the TOPS materials, to introductions to a more varied 
set of curriculum materials, some of which were TOPS materials (the 
Mini-computer boards, -the TOPS sourcebook, etc.); some were locally produced 

f 

problems, and some were conmercial materials. 



Distribution of W<?rkshop Topics and Ratings of Effectiveness. ■ (Percentages da not 
always total 100% because of teachers' responding patterns..) 



Overview Of the program's 
philosophy and goals 



Discussion/presentation of 
, ; math, cdntent 'by trainer 



Percent 
of time 

14% 



25% 



Demonstration/discussion by , 
■ trainer' of specific lessons 32% 



Practice by participants 
teaching vai^ous lessons 



29%- 



47* - 


,47%. 


6% 


effective 


somewhat 1 
effective 


not 
effective^ 


60% 


33% 


6% 


effective 


somewhat, s 
effective 


not 
effective 


66% 


1 • 33% 




effective 


somewhat 
.effective 


not , 
effective 


75% 


0 9% 


6% - 


effective , 


somewhat 
effective 


not 
effective 
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Thus, if a Coordinator. or Demonstration Teacher were available next year* 
almost all teachers would like demonstrations to be a first priority. Planning 
future ^lessons and discussing general problem solving strategies also were 
frequently cited by teachers. . 
4. Distribution of TOPS Lessons 

Site B's logs were designed to specify the lesson objectives and record 
the' problem solving strategy chosen from. a variety of strategies. Some of the 
teachers' strategies were traditional*problem solving activities and didn't 
represent the specific TOPS focus; in other instances it couldn't be 
determined which TOPS materials or strategies had been used or how the lesson* 
had been taught. ^ Therefore, logs were not relied on for any data other than a 
count of the number of times a specif ic, unambiguous reference to a TOPS 
activity was made. ^ 

Analysis of the logs' showed the following distribution of TOPS lessons. 



TOPS Activity 



This years Last years 
teachers teachers 



String Games 38X 34* 

Arrows T5% 1 5X 

Mini-computer 23* IS 

Hand Calculator T3* M 

Detective Stories 5% 

Geometry ' 2* S ' * • 

» ' *.'.'.. 
This, distribution of lessons asposs the strands, is roughly proportional to 

last year's distribution. However, far fewer identifiable TOPS lessons were 

recorded this year compared to' last year; the average number of identifiable 

TOPS activities per teacher was only about six. The decrease in numbers 
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reflects the site's decision to de-emphasize teaching specific TOPS lessons 
and use traditional forms of problem solving to teach mathematics topics found _ 
in the text books. 

5. Teachers' Assessments of TOPS Features 

« 

The statements below were generated from negative comments one or more 
teachers made during observations and interviews conducted in Spring, 1981. 
(Responses do not always total 100% because of the pattern of teacher 
responses.)' 



Strongly Strongly 
Agree Agree Disagree Disagree 
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Teaching TOPS takes too 
much preparation time. 12% 71% 18% 

TOPS ideas can be applied in 
situations outside TOPS 

classrooms. 35% 59% ' 



TOPS does not conflict with 
our schooTs math goals. 47% 41% 12% 

TOPS does not contain enough 
drill. Jl% J5% 18% 

Tests should be included as 

• part of the TOPS materials. 6% 41% 29% 

TOPS is noit useful in any 
other context. 47% , 

TOPS goals are clear. 24% 53% 18% 




TOPS does not help me learn 
more about my students 71% 29% 

The game aspect of TOPS 
lessens its value. 6% 18% 47% 29% 

Students who have had TOPS 
this year should continue 

with TOPS next year. 44% 56% 
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More teachers gave favorable ratings this year than last year,. and the 
ratings reflect generally positive attitudes. There were three negative 
statements agreed to by between a quarter and a half of the teachers; these 
three dealt with the need for more drill, more tests, and the .game aspect of 
TOPS. 

' However, -interview and questionnaire data may explain these three 
relatively negative sets of responses. While teachers expressed satisfaction • 
. with TOPS activities and materials, they also expressed' concern about spending 
time on TOPS at the expense of "local objectives". To many teachers, the text 
was a -set of local qbjectives to be covered as comprehensively as possible. 
.Teaching TOPS activities was viewed as conflicting in time and priorities with 
"the local objectives. Thus, while many teachers said they agreed with TOPS s 
go-als and could see the benefit, of TOPS-io students' performance they also had 
doubts about the wisdom.of not teaching the text — and risking lower test 
scores. -Hence the perceived need for more tSsts, more drill, and fewer games. 

6. Classroom Implementation Choices 

n 

t 

Since the site elected to change its TOPS program for Year II, the TOPS 
Sourcebook, a set of lessons developed for Year I, was not used as frequently, 
or in the same way, as it had been used in Year I. Electing an approach to 
problem solving which attempted to integrate the textbook and with many types 
^^f materials, the site left use of the Sourcebook to teachers' discretion. 
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Thus, teachers were asked how frequently they used 1) the district 's "text 
to teach problem solving, 2) the TOPS Sourcebook, 3) the local problem-solving 
activity sheets and 4) other commercially available materials. 

"In teaching TOPS, how frequently do you use each of the following?" 





38% ^ 


56% 






6% 


the text book 


always " 


almost 
always 


often 


\ rarely 


never 


the TOPS Sourcebook 


6% 


24% 


A CW 

45* 


1 ov 
10* 






always 


almost 
always 


otten 


rarely 


never 


"I Lovp TOPS" 


T2% 


24% 


40% 


24% 




locally produced 
activities 


always 


almost 
always" 


often 


rarely 


never 


Other commercially 


6% 


29% 


41% 


24% 




available products 
(please specify) 


always 


almost 
always 


often 


rarely 


never 


m 


7% 


7% 


- 36% 


21% 


29% 


Materials made by 
(please specify) 


■ always 


almost 
always 


often ^ 


rarely 


n6ver 



The most frequently used problem-solving material was the local textbook 
with ninety four-percent of the teachers using it "always" or "almost 
always 1 -'. In contrast, only 30% of the teachers used the "official" TOPS 
Sourcebook that often. The local activities and commercial products. were used 
with about the same frequency as the Sourcebook. 

Teachers' comments about teaching TOPS daily, as well as* their personal 
definitions of problem- solving, highlight the disparity which exists among 
teachers at Site B. One teacher said she made up her own activities based on 
ideas in the Sourcebook. She, teaches TOPS once a week for a full math period 
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and then twice a week devotes five minutes of the math period to word 
problems. Another group of teachers, four in the same school, reported 
different implementation strategies. One reported distributing the locally 
prepared checklist of problem solving strategies, which forms the basis of the 
teacher logs, to students. He advised them to apply the strategies to word 
problems and other math problems. The Mini -computer and String Game were 
never used in that- classroom, and emphasis on the thinking skills was 
restricted tq about ten. minutes each week. A second teacher used the locally 
produced activities in conjunction with the text because the two comp-1 emented 
one another, rarely used the Hand Calculator but did use the Mini -computer and 
the String Game. Another teacher reported using the text as the principal 
sburce of problem solving activities, and the last teacher used all resources 
available including commercial problem solving kits. 

A teacher in another school used Strings to discuss sets, and commented 
that while it was hard to-see TOPS' application to the regular curriculum, it 
was a "colorful review". Another teacher also reported she didn't use the 
Sourcebook extensively but instead "pushed mechanical techniques". 

These comments typify comments made by other teachers, i.e., some teach 
TOPS using the Sourcebook^seiectively, some tea^h TOPS using "a wide variety of 
materials, -some teach their own version of problem solving and use .their own 
strategy^ . 

- Teachers who did. use the TOPS Sourcebook were asked to rate its 
effectiveness* in teaching problem solving, its appropriateness for students, 
its'match a)" with the local textbook, b) with local goals, and c) with the 
teachers' 1 definition of problem solving. Ratings for three out of the five 
categories were ."average" to "excellent". Twenty-one percent of the teachers 

» * 

judged that lesson appropriateness was below average compared to other 



materials, and fourteen percent' judged it below average as a match with the 
local textbook. 

The questionnaire asked teachers if the Sourcebook needed revision, since 
it was anticipated teachers might not want to use it because of some major 
deficiency. In fact, a majority of l the teachers (71%) recommended that no 
changes be made. 

Teachers who recommended changes made four types of suggestions: 




o Add flow charts and schematic diagrams which had been developed 



during the June '81 planning seminar. 

o Explain every step of the demonstration problems and don't take 
for granted that the teacher knows how to do the problems. 

o Reference each lesson to. the standard curriculum, 
o Include more Arrow lessons. 
7. Changes In Teaching 

•One aspect of implementing TOPS is the' changes teachers made from Year I 
to Year II. Teachers mentioned several different changes from one year to 
another, tinlike Site A teachers who reported using questioning strategies 
more and encouraging students' analysis, Site B teachers discussed the 
Coordinator visits, improvement in their teaching, and integration with the 
regular text. 

Two teachers said that Year II -was not as productive for them because the 
Coordinator schedule was not as helpful as last year. , Three teachers said 
TOPS was easier to teach this year; having learned the problem-solving 
techniques they were able to practice those techniques this year. Integration 
of problem-solving techniques with the-standard curriculum was viewed as a 
positive step by teachers, but one teacher thought there was confusion about 
how and when to integrate problem solving with the text. . 
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One teacher reported using a variety of strategies and materials as a 
result of TOPS, another reported an increased awareness of problem solving, 
another^agreed with this and reported further that it was a direct result of 
'the teacher training provided by CEMREL staff during the workshops. However, 
other teachers mentioned that what. they had learned through TOPS was not 
general izalle since there were local goals and test scores to worry about.* 

' Teachers were also asked what changes they had made in their regular math 
class as a result of TOPS. Teachers replied (in about equal numbers) that: ' 

o. They didn't change since TOPS was so similar to their Own teaching 
style. 

o They had made no major change since TOPS doesn't fit the school, 
district's own curriculum and takes away from (regular) lesson 
planning. 

o They had added a variety of strategies (looking for solutions, 
analyzing the problem); had adopted a more open-ended questioning 
style, concentrated on training processes and not on answers, used 
a higher level questioning strategy and used basic math skills as 
a means of solving problems, not an end in themselves. 

8. Teachers' Views on Continuation 

Ninety-four percent of all teachers said they would be capable of teaching 
. TOPS next year and all teachers said they would choose to teach TOPS. 

While many teachers indicated they wjauldNnake no changes in the way they 
taught TOPS, "three said they would prefer to do it one day^ a week or every 
other week. Four teachers said they would try to integrate it more fully with 
the. curriculum, one would do more pre-planning and one would pay more 
attention to lower ability students. • 

• Asked what help they would need if they taught TOPS next year, two 
teachers reported they, would need no help, while others reported they would 
need support (5), supplies (3), ideas (3), and motivation (2).. 



> 

More than half of TOPS teachers said that at least 1S% of the math 

# 

teachers at their grade level could do an effective job with TOPS, and about a 
quarter said that 50% to 75% could do so. All teachers would like to see* TOPS 
used in their schools. * 

, "Teachers cited seven different problems they had faced in the past A 

♦ s 

including: 

o Adjusting to Coordinator's plans and schedule changes. 

o Understanding the program and how to use the materials. h 

o Relating the materials and activities to the curriculum. 

o Arranging the schedule in order to use the materials effectively. 

o Thinking of mathematics in terms of logical thinking and not 

r 

as computation, 
o Changing their teaching style to accommodate to TOPS, 
o Gaining confidence with TOPS. 
Asked what problems new-to-TOPS teachers would face, the same issues were 
stated again. Teachers who had taught TOPS said new teachers might experience 
difficulty because: 

# 

o There is not enough time for demonstration lessons, and time 
and practice are necessary if teachers are going to be 
successful with the project . 

o Many teachers do not have a strong enough mathematics background. 
They need time to become familiar with the materials and need help 
allocating time between mathematics as a reasoning activity and 
mathematics as -a computation activity. 

o Other teachers perceive the district's curriculum as fixed, as 
not accommodating TOPS goals, and therefore are unwilling or 
unable to change the curriculum and their approach to the 
/ curriculum. 
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In the course of implementing any new curriculum there are persistent • 
problems which teachers face. For teachers at this site,' problems fell into 
four categories: s 

o Coordinator visits were not frequent enough for teachers to . 
maintain momentum.* 

» %tt 

o Teachers found it' difficult to sustain their own or .their 
students' interest and motivation. 

o Time was needed. to learn the activities and plan ways to 
present them to the class. 

o Integrating the activities into the-curriculum required time, 
skill and understanding. 

For the Coordinators, persistent issues at the site -include a mutually 
agreed upon conceptual and operational definition of problem solving, time to 
continue training teachers to use problem solving since there are too few 
trained to make an impact on the entire district, and a way to integrate 
problem solving with the local text. 
9. Expansion Prospects 

0 

One of the district's goals for TOPS was the development of a cadre of'_ 
teachers who could act as change agents and expand the teaching of problem 
solving. 

The prospects for expansion are guarded. Like Site A teachers TOPS 
teachers won't be able to carry the entire expansion effort on their own. 
Some of the teachers from Year^I who might have been able to were not TOPS 
teachers in Year II, or were confused by the. second year* refocusing of the 
Voject. Too few were in the project both years and they were not adequately 
prepared to be a major force for expansion. 
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While the majority of teachers said they will use. TOPS materials next 

» 

year, they'also indicated that expansion within their schools and throughout 
the district must be directed by principals or other administrators. Teachers 
see themselves as having neither the time^nor the influence to effect TOPS 
expansion. Unlike Site A«where plans for" continued expansion and commitments 
for future funding have been made, Site B*s plans are not explicit at this 
time. 

However, the Coordinator and the Demonstration Teacher said the district < 
does have plans, including the part-time assistance of a Year I Demonstration 
Teacher- and the sponsprship of a course through the district's inseryice- 
agency. 

Aske^about expansion, principals said they had no plans for concrete 
plans or activities to promote the spread of TOPS in their building. One said 
current TOPS teachers would have to go it alone; the other looked to his TOPS 
teachers'to spread TOPS to other classrooms. This is interesting because 
teachers who were interviewed stated that theresponsibil ity for TOPS 
expansion must come from principals, central administrators, or both, if 
expansion is to occur. 

The logistics of expansion are a problem. Principals are, reluctant to 
hire several substitutes to cover classes so that current or future teachers 
can attend workshops. Consequently, expansion plans, if any, are compromised 
by the constraints of teacher training logistics. 

10. Overall Reactions of TOPS Participants : 

In an open-ended question on the questionnaire, teachers were asked to 
give an overall evaluation of TOPS. Teachers said that TOPS was an excellent 
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program and that they enjoyed participating. Several teachers said they will 
adopt its materials and techniques no matter what the district decides to-do 
because they appreciated its impact on students' thinking and found it helpful 
for their own development.' One teacher said it was an excellent way to meet 
NCTM's goal of teaching problem solving. The only negative comment was that 
it was difficult to find the time to teach TOPS activities and also teach the 
district's math curriculum. ' 

In sipmary, teachers' ratings of the workshops, the lessons, and TOPS 
features were positive:. However, some teachers are^ concerned about the match 

v * * * • 

between TOPS goals and local goals; For many teachers TOPS, in service and 
teaching was their first introduction to" conceptually oriented (as opposed to 
computationally oriented) mathematics. Those teachers are only beginning to 
gain a practical and conceptual command over TOPS. Expansion is taking place 
less rapidly than planned, and the district is commited to continuing and 
expanding TOPS at a much slower pace and in a less systematic *way than at' 
Site A. 
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'■"'.J- •' TOPS IMPAtTr^SI^E '6 

* * * \ • > < * 

* * » 

This section- is divided into two parts.. The/first part will present 
teachers' rating -of ft)P£ impact regarding: *' \ * ■ 

o TOPS effectiveness in improving students 1 - 'thinking skills 

% TOPS effectiveness in improving students' ability to do specific 

mathematical tasks ' . 

o TOPS value -for teaching strategies 

The second part wi-11 present students' standardized achievement test data. 

Teachers' Ratings of- TQPS Impact 

TOPS effectiveness .in improving students' thinking skills,. 

Much . Some No No 

Improvement Improvement Improvement Answer 

38% 56* • 6% 

. ca5U s ^.,.».., , 55 52 -i?r 

Evaluating Possible Answers 35% 59% 6% 

Reflecting Before Responding 29% 33 . 3S- #• 

Producing Multiple Answers 24% 222. IS. 2* 



Analyzing Situations 
Reasoning Logically 
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Teacher ratings were lower *for all categories this year. , 

* c 

1 

* TOPS effectivess in improving students' ability to do specific 
mathematical tasks. 

\ Much Some . No 

*> Improvement Improvement Improvement 

°i 

Place Value 2|* 50% lg 

Fractions • 1* 1* S£ 

Estimation , .14% 51% _ 29% 

Word Problems 50% 50% , • 

Decimals J[* |6% 23* 

Mental- Arithmetic ; 56% 1 44* 

Fewer teachers this year thought "Much Improvement" tiad occurred'. • 

8 
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TOPS value for teaching strategies. v 



Helps 1 Helps . Doesn't 
A Lot A Little Help 



Less pressure . 
(no* "wrong answers", everyone 

can 1 contribute) • ; 94% 6% 

Content * * 

(good problem situations, t / 
lesson plans) * 70% 24% 6% 

Student reponses * % I 

(explaining "how to" and "why",* - v - r 

\ many "possible answers) - 65% 35%* 

Hands«oiyi»ateriatn V ? . , } 

, (Minicomputer, hand calculator, 

-strings) c 88% h ]2% 

* % • 

Game atmosphere , 
(no paper pencil, team play) . 4 $8% 12% 

Teacher questioning * . -* 

(thought provoking, open-ended, * > 
follow up to student responses) . « 82% 18% 

This year, teachers 1 rating were slightly higher than last year's 
ratings for all but two categories^* content and student responses. 



Standardized Test Data, Fourth Grade Comparisons 

The primary goal of the TOPS staff and most^of the teachers was to 
improve students 1 ability to-solve word prdblems t in order'to iippnove 

I i • 

performance on 'the Concepts and 'Applications sub-test of the California 
Achievement Test which is routinely administered by the. district . Given that 

m 

goal, and the differences in the' way TQPS was implemented from or\e classroom 
to another* it seemed appropriate not to administer MANS Tests, but instead, 
to analyze achievement test data. « '.. " 
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The Sample*. Test data from seventeen classes of fourth grade students, tested 
in fay, 1982, were analyzed (9 TOPS and 8 non-TOPS). Fourth grade was 
selected because the largest number of TOPS classes were to be found'at Grade 
4. The TOPS classes tested were chosen by the site staff on the basis of 
their representativeness; most of the students were in .their first year of 
TOPS. Non-YOPS c-lasses were also selected by the site staff, the primary 
criterion for their selection being similarity to TOPS classes in student and 
teacher ability. ^ 

Method of analysis. For each class a mean score was calculated for each 
Mathematics test and for the Total Reading test of the California Achievement 
Test,. administered in the Spring of 1981. An Analysis of Covariance procedure 
was then used so that mean mathematics scores were adjusted to take into • 
account the differences in class ability, as measured by reading scores. 
(Appendix D presents TOPS and non-TOPS class means.) The mean Total Reading 1 
score (scaled score) for the nine'TOPS classes was 427.6 vs. 425.4 for the 
eight non-TOPS classes; thus mathematics scores were statistically adjusted 
(downward for TOPS, upwards for Non-TOPS) to reflect this difference. 

Results. Table 3 shows TOPS and non-TOPS adjusted mean scores. The p-value 
of the comparison is also given, i.e. the likelihood of a difference of that 
size happening by chance.^ 

TABLE 3 

SUMMARY OATA, CALIFORNIA ACHIEVEMENT TEST 
/ FOURTH 6RA0E CLASSES ^ SITE 8, 1982 





TOPS a asses 


Non-TOPS Classes 


p-Value 1 


Mathematics Computation 


420.7 


416.6 




Mathematics Concepts 
and Applications 


, 430.7 


425.2 


.35 


Mathematics Total 


424.9 


420.0 


.35 



1 8ased on the t-test for the Analysis of Covarfance with 14 degrees of 
freedom. A dash (-) Indicates the difference was not significant at even the .$0 
level (one chance 1n two)* 
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Although TOPS classes had higher scores on both subtests, the differences were not 
statistically significant. 4 
Graphs of Class Means. Data from Table 3 are amplified in the graph shaw^eTt>&, 
where Math Concepts and Applications class means are presented. The Math score, 
for each class is plotted against Reading score. A line has been drawn to show 
the best linear prediction of Math scores for given Reading scores. "It can be 
-s^rthat-Reading is a fairly good predictor of Math and that there is very little 
> advantage for 'the TOPS classes. . . 



fiWM OF CUM HEMS 
FOURTH tMK CLASSES 
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Standardized Achievement Tests, Second year TOPS Students vs. Non-TOPS 
Students . There is also a group of fifth grade students at site B who have 
had TOPS for two years. Although those students did not move in intact 
classes from one TOPS teacher to another, it was possible to locate groups of 
those students, and to record their Math scores from the "1982 CAT. In 
contrast to the TOPS students ( who were in small and large groups in their 
classes) the non-TOPS "comparison" group was composed of 'intact classes 
selected by the Coordinator because those classes were similar in ability to 
classes in which TOPS students were found. 

t 

It was also possible to record the 1981 CAT Reading score for these 
students and this was used as a covariate in an Analysis of Covariance, this 
time with students as the units of analysis. Table 4 shows the adjusted means 
in Mathematics for both groups of students and the resulting p-vaTue of the 
test of significance. The mean 1981 Reading scores were 431.1 for TOPS 
students vs.- 444.1 for t Non-T0PS. s 



Table 4 

Summary Data, Second Year TOPS Comparison 
Fifth Grade, Site B, 1982 

TOPS ajid Non-TOeS Students 
1980-1981/ 1981-1982 





*tops 

Students 


. Non-TOPS 
Students 


p-Value 1 * f 


CAT Mathematics 
Computation, 1982 


469.9 


462.6 


.30 


Mathematics Concepts 
and Applications, 1982 


470.4 


461.4 * 


.10 


Mathematics Total, 1982 


468.6 


460.5 


.10 



Based on the Analysis of Covariance t-test with 211 degrees of freedom. 
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The differences, though favoring TOPS students were not-significant at 
the usual^.05 level, but^ were significant at the .10 level (.i.e. a one-in-ten 
likelihood this difference occurred by chance). 

One should bear in mind that this analysis had to done at the student 

level, yielding more liberal results, than the more usual class level 

analysis. Rather than grouping students into "classes, the graph below shows. 

the results when students are grouped according to reading ability- Thus all 
* * 
. students with Reading percentile rank between 75 and 99 were grouped together 

and average Reading and average Math scores computed for that group and 

plotted and similarly for the other quartiles. This yielded graph points* for 

TOPS and for non^TOPS students for each quartile. Because TOPS schools 

generally were selected on the basis of low achievement scores, these points 

are always based on progressively more students in the lower quartiles (e.g. 

- for TOPS students, 10, 20, 24, and 48 students respectively. r 

TOPS VS NON-TOP % m QUARTItf MEANS 
FIFTH GRADE 
(X» TOPS ClfSS, •* Non-TOPS Ctu*j 

CONCEPTS AND APPLICATIONS SUtTEST ' 




LLjAU 

360 X580 



420 440 460 480 
READING COMPREHENSION 



500 



520 
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It can be seen that it is in the middle quartiles (i.e. percentile rank in 
reading from 25-75) that the .TOPS students do relatively best. 
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RECOMMENDATIONS <i 

s 

The data show that TOPS was favorably received by the majority of 
participating staff and had some effect on participating students. The 
favorable reactions speak well for both CEMREL staff and the local -districts' 
staff. Their work with TOPS was clearly a key factor in teachers' evaluations. 

The next question is "What do potential adopters need to consider given 
TOPS history at Site A and Site B? 

TOPS history points to several major conditions (in our view) which are. 
necessary for success* To the extent that present sites are meeting these 
conditions they will be successful; future adopters will need to review these 
conditions with an eye toward their own situation. 

o Supervisors and teachers should know enough mathematics to understand 
the mathematical processes in- TOPS activities and know how the lessons lead to 
students' understanding. 

o If teachers have Sufficient mathematics background to understand and 
teach TOPS, they should be able to play the major role in implementing TOPS in 
their classroosm. Coordinators would*then act to analyize, review, and 
critique. 

o If teachers do not have sufficient mathematics background, the trainer 1 
will have to play the major role in demonstrating and discussing lessons; the 
mathematical processes embodied in the lessons will need to be explained, and 
the goaT of the lessons specified. For these teachers it would be a mistake 

* 

to assume that the lessons are self-explanatory but it would be equally 
undesirable to-teach mathematics first and then teach TOPS activities.* 



Throughout the year the trainer may also need to play a major role in the 
classroom, first demonstrating lessons and then critiqueing teachers' les'sons. 

o Both groups of teachers, the mathematically knowledgeable and those 
less so, should remember to begin teaching both TOPS activities and 
traditional problem solving methods at the same time m.ay confuse students, more 
than help them. Once sol idly, in place, TOPS may be a springboard for 
introducing traditional problem solving, but given TOPS' specific point of 
view at the start, the best way to maximize its effect is to learn and 
practice TOPS lessons. 

o Once the decision is made to adopt TOPS, a district must provide 
sufficient training initially, and follow up for teachers. If a long term 
impact on students is desired, then a large enough cadre of teachers should be 
trained so that TOPS can be implemented at many grade levels and year-to-year 
scheduling can be simplified. Training specialist teachers, who teach more 
than one grade level^is advantageous for long term effects. Training 
specialist teachers is also a more cost effective way of reaching the largest 
number of students in one day, which maximizes the training investment. 

Teachers need to be told the district is commited to TOPS. Since the 
lessons might not seem to directly translate to achievement test success, the 
district needs to communicate a willingness to bide its time and to delay a 
.final weighing of the evidence Lfl^til a comprehensive implementation .has 
occurred. Otherwise, teachers are likely to return to what they perceive to 
be the district's "real" goals. 

There are several caveats for potential adopters. Learning the TOPS 
activities may frustrate the teachers at first, and they should be encouraged 
to persist. Coordinators are the key in this respect. They will need to ' 



monitor how often and how well TOPS is taught. Then, the evaluation of TOPS 
will need to be carefully considered. "Traditional" achievement tests may not*, 
be sensitive to TOPS instruction and districts should be .aware of -the 
difficulties inherent in assessing problem solving. s 

Oistricts whose goals match TOPS- goals, and who have the resources to 
train teachers and monitor their TOPS implementation, should find TOPS makes a 
positive contribution to teachers and students. 
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Appendix A 
Questionnaires 
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TOPS TEACHER QUESTIONNAIRE 

School: 



I. Background 

1. How many years have you taught? 

□ ist year Q2-5 y* 4rs O 6 '* 0 years C' 0 * 20 * ears C2 20 p1us years 

2. Highest academic status attained: 

Course work leading Bache ^ or . s Bachelor's M|Ster , s Master's 
to a Bachelor's MW * u plus . plu_s 

□ □ □ □ c 

3. In college, did you elect: 

Elementary Education Elementary Education Another major with Concentration 
Major with no math with 1 or 2 math x some math courses in mathematics 

methods courses methods courses 



□ □ n 

4. What" grade level (s) are you teaching math to this year? 

5. How many classes do you teach math to each day? 

6. How many years have you taught math at this grade level? 

L What other grade levels, if any, have you taught math? 



8. On an average, how many minutes does your class spend on math each day? 

9. Did you volunteer^ or were you asked to participate^ in TOPS? 



10. When did you begin teaching TOPS? 

During the previous school year. At the beginning of Within the last few months. 

■ present school year. r 

□ G □ < 



II. Teaching TOPS 

1. Since you started teaching TOPS, what percent of math time has been spent on TOPS Instruction 
Is 1t usually for: a few minutes at a_t1me, but not every dayf"""] 
a few minutes at a time each day | 
- a ful-1 math period 'each weekj | 

d A-l ' ft < 



If yow are a second year teacheff briefly compare teaching TOPS this. year to teaching 
TOPS last year. 

S ■ 



3. Check whether you strongly aqre agree , disagree /or strongly disagree with the 
following statements: 



Strongly ' Strongly 
Agree Agree Disagree Oisagree 



Ttaching TOPS takes too much ' 
preptration time. ,# 

TOPS ideas can be appHtd 1n 
situations outside TOPS classrooms. 

TOPS dots not conflict with our school's 
■ath goals. 



TOPS does not contain tnough drill. _ _ 

Tests should be Included as part of 

the TOPS materials. 

* 

TOPS is not useful in any other context? * — _ 

TOPS goals are cltar. _ 

TOPS does not help me learn more about 

my students. * . — — 

The game aspect of TOPS lessens its 

value. ' ' — — 

Students who have had TOPS tnls ytar ' , 
should continue with TOPS next year. 



4. What changes, if any, have you made in tfte.way you teach or what ycu teach during 
your regular math. class? (state briefly) 



Ill- Continuation 

U As i result of your present TOPS experience , if you hid the TOPS materials but the 
coordinator was unavailable 

a) would you be capable of continuing to teach TOPS? YesQ NoQ] 

b) would you choose to teach TOPS next year? Y « S Q No O 

If yes, what changes, if any, would you inak*€? 



If no, why not? 



2. If you did teach TOPS next year, what help, if any, would you need? 



3 # About what percent of teachers now teaching Math at your grade level (s) could do an 
effective job with TOPS? ' , 



4. Would you like to see TOPS used in other classes in your school? 

No Pj Yes, at all grades ["") Only in grades Pj 

5. What is the biggest problem new-to-TOPS teachers face* in teaching TOPS? (comment briefly) 
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IV. Workshops and Coordinator Activities 
1 



During the workshops what percent of the time was. spent on eich topic and how effective was the 
presentation? 



Percent 
of time 



Overview of the program' s 
philosophy and goals 

Discussion/presentation of 
math content by trainer 

Demonstration/discussion by 
trainer* of specific lessons^ 

Practice by' participants 
of teaching various lessons^ 




effective 


somewhat 
effe cti ve 


not 
effective 

i 


1— , , 

effective 


somewhat 
eff ecti ve 


not 
effective 

i 


effective 


somewhat ( 
effective / 


not * 
effective 

not 



effective 



effective 



2. What were the strengths and weaknesses 
a) of the workshops 



ERLC 



b) 6f other coordinator activities (i.e., individual contacts, meetings) 

/ . • 

If a coordinator were available next year, what activities would you like to see 
emphasized? - (check as many as apply and rate the helpfulness of each; 



very somewhat not usually 

helpful helpful helpful 



Conducting demonstration classes 

Reviewing mathematical content 

Planning future lessons 

Discussing general problem solving strategies \ 

Critiquing lessons you taught 

Explaining the goals of an individual lesson _ 
Suggesting classroom management procedures — 
Classroom. visits in general 



What were 



the most persistent problems in implementing the program this year? 



Benefit to Students ' 

1. Howwell does TOPS improve students 1 abi1ity\to do problems in: 

much ' some no ' 

(check one) improvement improvement improvement 

piece value 

•fractions — — — — — 

estimation ^ — p 

word problems — - 

decimal s ' - — — 

mental arithmetic 

2. What aspects of TOPS make it beneficial to students: 

(check one) 

Helps Helps Ooesn't 

a lot a little help 

Less pressure"" . . . * 

(no "wrong" answers, everyone can con tn cute j 

. Content . 

(good problem situations, lesson plans) 

Student responses 
(explaining "how to" and "why", many 

possible answers) — 

Hands-on material — m4t . „x 
(Mini-computer, hand calculator,- strings) 

Game atmosphere 

(no paper pencil, team play) — 

Teacher questionning 
(thought provoking, open-ended, 

follow up to student responses) — r~- 



3. How well does TOPS improve students' thinking skills in: 

^ . (check one) 



reasoning logically 
evaluating possible answers 
reflecting before responding 
producing multiple alternatives 
analyzing situations 



much some n ° 

improvenent Improvement -improvement 



Site A only 



VI. TOPS Materials 

1, Rate these 

as hign 
or low on: 



TOPS materials-- 


String 


Language of 

Arrows 


Hand 

Till nil ator 


M1n1- 
CGmouter 


Oetective 
Stories 


Can be used 
effectively 
by a teacher 






1 






Promoting Student 
growth in problem- 
solving skills 






V 


• \ 




Promoting positive 
student attitude 
toward problem- 
solving 












Student enjoyment 










o 


Overall benefit to 
students .compered 
to the math period 
it replaced 






0 







2. What changes, 1f any, would you recommend be made in any of the TOPS material s. 



VII. 1. What assistance have you provided to other teachers in your school f 1f any, who have recently 
started TOPS? 



41 



VIII. 1. Overall* what is your view of the TOPS program? 
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• • Site B only % 

VI. TOPS Materials 

t . \ n teadHno TOPS, how gently do you use etch of the foiling: 



t^t text book 

the TOPS Sourcebook 
(orange TOPS book) 

-I- Love TOPS" 
Activities 

» 

Other conwerdallv 
available products 
(please specify) 

Materials made by 



always 



almost 
always 



often" 



rarely 



always 



almost 
always 



2. Are there any changes you 

YesQ 

If yes, what changes? 



would reconwend be made 1n the TOPS sourcebook? 
Nog 



J 



never 




never 



„ ,. u ~ m TOPS Pl.»» «-N« « ■ 7 f1 " S 

Ease in Teaching Problem Solving 



Poor 



Below 
Average 



Match with Local Textbooks „ 
Match with Local Goals 

Match with ycyjr Own Definition of 
Problem Solving 



■Average SBove Excellent 

Average 



Poor 


Below 
Average 


Average 

■ 


Above 
Average 


Excel lent 
Excel lent 


Voor - 


4elow 
Average 


Average 


Above 
Average 

\ 


Excellent 


hor 


Below 
Average 


Average 


Above 
Average 

■ 


Exceilenr 


tobr 


Below 
Average 


Average 


Above 
Average 





VII. 1. ^Overall, what 1s your view of the TOPS program? 



( 
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o "Overall, what is your view <£f-T0PS?" 

Site A | 

It would make a great elective. _ 

I'm not certain that a loss of class periods is more beneficial than 
math. I did not accomplish .what I hoped to this year. 

Very -good. . 

I like TOPS. I had a bad start last year. Now' I expect to get into" 
the program. 

It appears to pr'omote problem solving techniques and can be used in 
conjunction with any math program. 

Excellent. Problem solving skills can be developed using it. 

I- believe I've seen improvement of thinking skills and it's spilled 
over into classroom activities. 

■ A challenging program if you can .attune students tq it. It is 
difficult to get them to see it is math. 

I compared students' scores and some went up and some went down. 
It is a rewarding experience. 

It is an excellent program to get students to think. They're all 
eager to be involved. 

It's great. It's too bad this is necessary beyond the primary grades. 

Terrific. It affects the cognitive domain of -students. It is a much 
needed program. 

. It is excellent. 

I'd like to see a follow-up for teachers to show TOPS supervisors 
how a lesson they would teach could be put in a TOPS format. All 
teaching could be done that way with careful planning. 

I'm impressed. I use it twice a week. The style, questioning and 
reasoning together, works well. It spins off in the teaching of 
other math materials. • ' 

A great first step to improving math abilities. We should not 
stop here. 

Overall growth at the end of the year in some areas. 



ERIC v 



It tes not been a cure-all. I'm not totally satisfied with it, maybe - 
. because I used it in .only one class. ' 

It is excellent~*in developing problem solving in all areas of math. 
It has made a difference to my attitude and approach in teaching math. 

Overall, well done and good materials. 

Essentially beneficial. . 

Great. It's too bad this kind of analysis training is necessary _ 
^.beyond the primary level. 

It's rood for students 1 reasoning skills.- There is too much emphasis 
on test spores. Reasoning is being neglected and too many students 
know^how to add fractions but in real life are not able to apply it. • 

r think it is an excellent tool, especially for use in teacher 
training. Sell it to ^teachers' colleges first. > 

Great,, if there is time. Curriculum demands make it difficult. to 
use it much. 

te B 

Excellent concept in keeping with the problem salving challenge of 
the eighties. 

1 **I lik£ it generally, but as an auxiliary to the^regular text and 

curriculum. s 

Jt is worthwhile teaching for problem solving. I'm glad I had the 
experience, of working with it. I hope pupils will benefit. It 
/ will be part of my program. 

_. v «« 

Pretty good. l*plah to use it more next year. 
'♦I enjoyed -it. It was helpful to students. 
r I "love it. It gears on w thinking skills. 

\Good. It is helpful to feachers and pupils. It* meets the needs of both, 
itfl was more at ease, I might teach it more. 
Excellent.' 

. 

Overall, goc^.~ It should staft at grades 2 and 3. The workshops and 
ma th v (cbn ten t\ seemed aimed at junior high school or higher. It needs 
•jnore emphasis to Iswer grades. * 

I enjoyed participating and being exposed to activities.,. I will use it. 

I enjoyed it. ... 

I thought it^was great and will ^miss people. • 

Excellent program'.* It lessens the proElero of teaching problem solving. 
, Problem sol ving^'tpuld be .improved in all areas of education". 
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The- MANS Tests 

The MANS tests are short test scales originally designed to assess what 
are thought to be some of the underlying thinking skills of CSMP, an 
experimental mathematics curriculum developed by CEMREL, INc. They have been 
used in the evaluation of CSMP for several years in second through sixth grade 

studies. - 

Characteristics of the MANS tests include: x. 
~. Testing of important mathematical skills 
. Situations presented are usually new to students 
. Do not contain non-standard terminology 
. Series, of short scales, each preceeded by special 

m 

t % 

directions and samples 
~~ . Provide 'for a mixture of multiple choice, open-ended, 
.and multiple responses 

, '-*S 

. Require little reading., and when reading is required, 

the reading level is well below grade level 
. Low algorithmic computational requirements 
. Main unit of analysis is the classroom 
For all MANS scales, an intensive pilot test and review procedure is used 
during the development phase. CSMP Evaluation Report $-B-3 contains a detailed 
description<of this process for an earlier CSMP study. 

A brief description of each test is given below. Similar tests have been 
grouped into categories of mathematical content. 

Administering the MANS tests requires trained testers, who were given 
specific directions for administering the tests, as well as a standardized -* 
script wr.ich included Sample problems from the scales. During the testing v 
session students work a sample with the tester, then do the items in that 
scale- on their own. 



MANS tests are, administered to intact classes, in two separate sessions 
'which are usually scheduled on two consecutive, days. Each session lasts from 
45 to 55 minutes. Some scales, like Estimation, are strictly timed; for others 
tjming is flexible* . . 

Four categories' of scales were administered to TOPS classes: Estimation 

and Mental Arithmetic, Fraction and Decimals, Represenations and Patterns of 
Numbers anr 4 Elucidation. In all, twenty scales were administered, the scales 
averaging 11 items per scale. Item sampling was done for most scales. A 
description of each category and a sample from each scale are shown below. 



• ( ■ 



\ m 



ERIC 



CATEGORY :> ESTIMATION, AND MENTAL ' ARITHMETIC. ■ ' ... 

Includes two, types- of scales: Estimation requires the. rapid deriving of approximate 
answers, to problems; and Mental Arithmetic requires the exact answers to calculations 
amenable to non-algorithmic solution. " m - 

15 scales,- 66^ i terns ••" 



E*. Estimating' Intervals: Addition 

Abstract: Given a computation problem Involving, whole number addition, and 
5 fixed intervals (0-10, 10-50, -50-100, 100*500, 500-1000), 
determine which Interval contains the answer to the problem, and 
put an x in the Interval. 8y Instruction, format and short time 
limits, students~are discouraged from computing exact answers. 
Two or three sample Items are done collectively. 

Specifics: 8 items, on one form taken by all students, exactly 1.75 minutes. 

Examples: 479,+ 85 0 1 0 80 Jf 100 500 1000 

19 * 19 * 19 * 0 10 50 100 500 1000 



E2. Estimating Intervals: Subtraction 

Abstract:* The scalers similar to El (except that It Involves whole number 
subtraction) and follows it directly in the test booklets. 

Specifics: 8 items, on one form taken by all students, exactly 1.5 minutes.' 

Examples: 100 -* 93 0 10 50 100 500 1000 

765 - 217 0 ' 10 50 100 500 1000 



E3. Estimating Intervals: Multiplication m ' 

Abstract: The scale 1s similar to El and~E2 (but 1s devoted to multiplica- 
tion with whole. numbers for the mosfpart) and follows them in 
the test booklets. 

Specifics: 8 items, oh one form taken by all students, exactly 1.5 minutes. 
Examples: 40 x 10 0 10 50 100 500 1000 
4 x 29 0 10 50 100 500 1000 
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E4. Estimating Intervals: Oivision 

Abstract: The scale is similar to E3 except that it involves division, 
mostly of whole numbers and there are different intervals 
(0-1, 1 -10, 10-20, 20-100, 100-500) in- the response format. 
It follows E3 in the test booklets. 

Specifics: 16 items, 8 on each of two forms, exactly 1,5 minutes. 



E5. 



Examples: 



190-5-10 
18,230 + 1,000 
Whole Number Open Sentences 

Abstract: 



0 
0 



10 
10 



a 

20 



100 

-100 



Given an open sentence, where the box may be either on the right 
or the left of the. equal slgnt where the numbers are large and 
easy to work with, and where only one operation 1s used, put the 
number 1n the box which makes the*sentence true. By Instruction 
and prompting, students are discouraged from "computing the long 
way* and are not allowed to do any figuring on paper. 



Specifics: 24 Items, -12 on each of two forms, exactly 7 minutes, 
one Items are of Type A, three are of Type B. 

Examples:. Type A: 500 + 9,000 + 500 ■ 1 I 

□ - 250 ■ 150 

12 x 50 ■ □ 

900 divided by □ » 3 



Twenty- 



Type B: 



7 x 43 ■ 301 
14 x 43 ■ 



600 divided by 35 
615 divided by 15- 



40 

□ 



-E6. Negative Hits and Misses 



Abstract: Given the description of a "game" with two rubles ( a} each hit 
means a gain of 5 points and b) each miss means a loss of 1 
point) and. partial Information on turns, the student must deduce 
the m1s$1no Information. Two sample Items are completed 
collectively. 

Specifics: 10 Items, 5 on each of two forms, approximately 4 minutes. = 



Examples: 

Jan: 
Peter: 



Started with 
a score of ^ 



Number 
of Hits 



3 above zero 



10 below zero 



CD 



Number 
of Misses 

□ 



Endtd with 

a scort of = ^ 
=- • 



= Atom art 



12 below zero 



HP 



9 
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E-I *» 

(provided, 
but not 
mentioned 
in Instruc- 
tions) 
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CATEGORY: FRACTIONS AND DECIMALS * 

Requires computation with fractions and decimals, in a variety of novel applications. 



6 scales, 60 items r 



F1. Decimal, Gas - 

(+,.'-. x,. or f) with decimals. 
Specifics: 8 items, 4 on each of two forms, approximately 3 minutes . 

£xamples: Peter has 6.5 gallons. 

Then he spills 1.2 gallons. 

How much gas will he have left? 



Ron has 6.5 gallons. 

Next week hewljl use ten times this much. 
How much gas will he use next week? 



P2. Representing Fractions and Oecimals 

■ Abstract: The scale has five short subsections each containing one of two 
kinds of Items: either .a mixed number or decimal is given 
1n standard form and must be represented 1n another specific' 
way or else that process is reversed and the response format" ' 
is multiple choice. Instruction 1s largely 1n the form of a 
written question or command at the beginning of each subsection. 

Specifics: 20 items, 10 on each of two forms, approximately 4 minutes. 
Examples: fcit m «rrow «t 1.3S inches. jiiiijnujini prrT|TnTpTrrrrrnpTrrj 

0 1 2 3 4- 



1 i 2 

How audi Is sludtd? ^ J ? 7 nont of tt,s * 

* » 

A 

(A completed sample wajs-glVej. ) 
F3. Fraction Computation • 

Abstract: Given straightforward computation item Involving simple 

fractions, produce exact answers (by circulating on paper if 
necessary). Though the items do not hive the multiple choice 
response format, they are s1m1lar"Tn range and difficulty to 
those found 1n the standardized achievement tests of the appro- 
priate grade level. \ 

Specifics: 10 items, 5 on each of two forms, approximately 1 minute. 
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F4, Fraction Open Sentences 



Abstract: Given an open sentence involving at lea ?\° a ne h f a r ^j°"' e and one 
of the" four arithmetic operations, complete the sentence. 

Specifics: jMtems,-on one form taken by half the students, approximately 
"3 minutes. ^ mmm ^ 

Examples: I * O ' ' • ^ f " O " 7 



F5. Which Fraction is Larger 

Abstract: Given two non-whole numbers written in fractional form (a proper 
fraction, an improper fraction or a mixed number), circle the 
; larger one. A completed sample item is shown. 

Specifics: 6 items, on one form taken by half the students, approximately 
2 minutes. 

Examples: y or lj 



4 



or 



5 

T 



F6 Which Oecimal Is Larger 

fractional form. 
Examples: 4.999 or 5.1 



1.5 or 0.58 



• u 
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Requires finding or applying a given pattern in sets of numbers. 
6 scales, 72 items 

Rl. Labelling Number Lines ■ • '. - * 

Abstract: Slven partially labelled number lines, with varying Increments, 
determine certain missing numbers. A sample Item 1s worked 
collectively. 

Specifics: 10 items, 5 on each of two forms, approximately 5 minutes. 



Examples: 



,4 j±j .. . 35 



— r 

49 



T 

2 



6 



J 

14 



— r 

23 



R2. Multiplication Series - 

• Abstract: Given an Incomplete portion of a 1? 1 ?] 1 ca ? 1 ve , s i!I i ° n L„ frt 
numbers, determine the constant multiplier involved n order to 
complete the portion shown. Portions of several series are 
shown altogether with one, two or three numbers missing from 
each. A sample series 1s examined and completed collectively. 

Specifics: 14 Items (missing numbers), 7 on each of two forms, approximately 
3.5 minutes. 



Examples: 



| 1 j | | l.ooo I io>floo |ioo.ooo j 



1 



• • • 



it 



• • • 



R3. Constructing Numbers 

Abstract: S1ven the use of only four digits (2, 5, 7 and 8) and the rule 
that no digit be used more than once, construct numbers like the 

... smallest (or largest), the second smallest (or largest) ,or the 

closest to a given number. The constructed numbers are to be of 
either 2, 3 or 4 digits and sometimes restricted to a given 
range of numbers** Collectively, to clarify the rules, two 
Incorrect answers and the correct one are examined for two 
sample problems. 

\ . 

Specifics: 12 Items, 6 on each of two forms, approximately 4 minutes 
Examples: What is the second largest four digit number? 



What 1s the smallest three oMglt number between 

730 and 850? 



What four digit number between 2.0Q0 and 3,000 1s 

^closest to 2,800? 



,4. 



Attributes of Flags • * 

Abstract- Given drawings of six flags which have different combinations of 
Abstract. J f , which is described by a statement. 



Specifics: 
Examples: 



attributes, select the flag 
12 items, 6 on each of two forms, approximately 6 minutes 





Bttft 11 lets flags that don't navt a strips or a star. 
UMch fl*gs would sh« 11k*f 
ClrcU *11 ti» corrtct answtrs.- 

A 8 C 0 E 



In wtut w»ys »r* fl*gs 8 u\d F *Hlct? 



R5. Solving Number Rules 

Abstract: Given 3 clues (I.e., pairs of numbers) 1n a game, determine what 
* the secret method 1s (I.e.* the unique rule relating each of the 
pairs of numbers) and. then use the rule to calculate the missing 
number from the fourth pair. 

Specifics: 12 Items, 6 on each of two forms, approximately 6 minutes. On 
half the Items, 1t is the second number that" 1s missing from the 
fourth pair; on the other half it's the'f 1r$t one that 1s 
missing. 



Examples: 



Amy's Game 



Jane's Game 



First clue: 
Second clue: 
Third clue: 
Question: 



Class 
said: 

5 

9 

2 

4 



Amy's 
answer: 

26 

46 

11 

□ 



Class 
said: 
9 

15 

30 

□ 



Jane's 
answer; 

3 

S 

10 

, 7 - 



R6. Using Number Machines . 

Abstract: Given labelled- -number machines- 1n sequence and either the 

Initial or the terminating number, determine the other number. 
There 1s an Introduction showing that "number machines take In 
numbers; add, subtract, multiply or divide by a fixed quantity; 
and give out the resultant number. Then three sample Items 
(each with a "number machine" sequence) are worked collectively. 

Specifics: 12 items, 6 on each. of two forms, approximately 4 minutes. . 



Examples: 



+ 2 



+ 2 



TO 



Ay 



x2 
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CATEGORY: ELUCIDATION ^ 

Requires producing many correct answers to a given problem. 

2 scales, 30, 34 possible ^correct answers 
Ul. Getting to 12 

Abstract: Given a starting point (0), a goal (12) and two rules, invent as 
many ways of reaching the goal as possible. The rules are that 
only the numbers, 2, 3, 5, a 7 can be used along with addition, 
subtraction multiplication or division. Two sample solutions 
(see below). are worked collectively. 

Specifics-: Exactly 3.5 minutes. 

* » 

Examples: - , 

Example 1: 0 +2 " 7 

— 7 x 2 * 14 

— Mil -12 

Example 2: 0 +_5 » 5 

5+3*8 

8+2*4 

4 x 3 » 12 

U2. Producing Many Answers 

Abstract: 6iven several different situations each of which poses a problem 
for which there afe many correct solutions, produce as many of 
them as possible.;, For each situation, some potential solutions 
are accepted or rejected for not following the given rules as 
inappropriate. . 

Specifics: Four sJKTa,tions; only the first one is read by the tester. All 

studefts Jo the first 'two of four; do the remainina two on Form A; 
- half do~tne remaining two on Form B. Total of 17 possible correct 
solutions to the first two and for the remaining two items 
-fourteen correct for F6rm A; eighteen correct for Form B. 

Examples: K\ f% - . tun » nn. 



3^Ltt ! fttfton. ^ 

Huy Mt It ttvUUIt If I. ^ 
Utt? Mi It *m)Ur Vm M. 1 

¥n til tut ormt nirnn. 10. 
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List of Class Means, Site A 
MANS 'Categories 
TOPS/Non-TOPS 



CI asses 


Estimation 
And Mental 

Ari thmpt i c 


-•Fractions 
and 

Decimal s 


Re Dre s en t at ions 
and Patterns of 
Numbers 


Elucidation 


Total 


Figural 
Reasoning 


1981 
Reading 


TOPS 1 


34.7 


27.4 


40.5 


32.9 . 


135.8 


7.8 


24.4 


N 2 


31.2 


25.8 


37.9 

c 


24.4 


119.3 


8.3 


17.4 


3 


31.9 


24.2 


31.8 


12.9 


100.8 


6.6 


17.4 


4 


o 

33.3 


21 .2 


35.2 ■ 


17. Z 


i uo«y 




1 0« 0 


5 


45.2 


41.9 


48.6 


39.4 


175.1 


9.1 


' 24.8 


u 


33 0 


22.7 


' 34.8 


16.2 


106.7 


7.7 


14.6 




27.8 


20.1 


27.6 


19.3 ■ 


94.7 


7.6 


22.5 


2 


41.1 


28.0 


39.5 


24.7 


133;2 


8.4 


17.2 


3 


28.7 


21.7 


31.5 


16.6 


98.5 


' 7.7 


16.1 


4 


29.0 

• 


24.9 


32.$ / 


17.0 


103.4 


o. 1 




5 


38.0 


32.4 


40.5 


32.3 


14,3.3 


8.6 


23.3 


6 


28.5 


20.2 


32.5 

«■ , 


33.0 


114.3 


8.5 


22.0 
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List of Class Means', Site. B, 
California Achievement Tes<ScoVes 
- 1 1981-1982 Students 

' i 



\» " *" Classes 4 Computation 
•• 


Concepts and 
• Applications 


Total 
»* 


Reading 

** 


. TOPS 




* * 


• t 




e 


1 


385.2 ' 


407.9 


397.0 


417.0 




2 


451 .'4 


AA7 1 


AAA c: 

440 .0 


AR7 7 




3; 


A ftrt ^ 


4*C£.0 


^ OO* 7 * 
4c£. / ^ 


41 d.y 

• 




4- 


399.4 


421 .2 


410.5 


410.5 


... » 












* " \ ■ 


5 


4"fe.9 . 


A^ 1 

* 


A 9C 1 






.6 


ill ^ n 

415.9 


415.3 


* 415.1 


412.4 


1 • *-•/••• 




425.4 


435.8* 


430.6 


437.9 ' 




* 

8 „ 


. 407.9 


*T 1 0 • 0 


*f 1 1 .4 


oy 1 .4 


• > * 

* • — 


9 


, 459.4 ' 




^71 O 
4/ 1 . 0 * . 


/I qa n 
4o4.U 


Non-T0PS v 












1 


401 .8 


aAe 0 

405.8 


404.3 


436.1 






- 439.9 • 


446.9 

« 


A A T f\ 

441 .0 


A Tf ft 1 

478.1 * 


1 


3 s 


- 418.5 


438.9 


A ft^ "7 

427 n 7 


« ft'l y ft" 

431 .2 




4 


37a. 6 " 


< 




100 0 




5 - 


»418.?" 


430. r 

# 


424.2- , 

• * 


418.6 




6* 


- T . 409.1 


417.4 




406.5 




7 ; 


442.4 


" 440.3 •'. 


439'. 1 


447, 2 




•8-* 


418.0 ; 


423.2 . 


419.0 


396.1 















List of Class Means, Sdte B 
Fifth Grade Classes 
TOPS/Non-TOPS 



Computation 
TOPS Class 1980-81 1981-82 



Concepts and 
Applications 
1980-81 1981-82 



Total Math Reading 
1980-81 1981-82 ' 1980-81 



04 
04' 

__-Q8 . 

v. 

13 
14 
16 



418.1 

475.1 

3&o.D 

398.5 

425.2 

411.6 



433.7 
499.8 
'34.7 
434.8 
474.2 
469.1 



415.0 
444.2 
402.0 
423.9 
431 .8. 
429.3 
W 



432.3 
508.3 
.423.8 
454.1 
465.7 
463.0 



414.6 
478.2 
396.2 



432.1 
502.3 
428.6 



fll.5 444.6 



427.4 
419.7 



467.4 
464.4 



399.9 

• 

482.5 
364.6 
414.6 
428.2 
435: !9 



Non-TOPS 
CI asses 



01 
07 

18 



471.9 


438.2 


•464.5 


435. 5^ 


465.6 


437.9 


443.4. 


*423.1 


432.0 


415.3 


436.2 


424.3 


* 












431 .6 


4^2.9 


> UZ.2 


41 5. 9 


436.7 


42J^T 


515.8 


464.1 


523.4 ( 


455.9 


517.8 


484.9 

r 



f 



. ■) 
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